NormAct: A Benchmark for Hidden Social Norm Compliance in Embodied Planning

• URL: http://arxiv.org/abs/2606.27826v1
• Date: Fri, 26 Jun 2026 08:10:39 GMT
• Title: NormAct: A Benchmark for Hidden Social Norm Compliance in Embodied Planning
• Abstract summary: We introduce NormAct, a benchmark for embodied social-norm interactions.<n> Experiments show that models achieve explicit goals in 67.3% of cases, but comply with hidden norms in only 26.4%.<n>We propose NormPerceptor, a context-conditioned cue generator that infers scene-relevant norms prior to planning.
• Score: 15.111667090272809
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal large language models (MLLMs) are increasingly deployed as embodied planners in egocentric environments, where task success requires not only achieving instructed goals but also acting in socially appropriate ways. While explicit goals may render certain actions optimal, implicit social norms often impose hidden constraints. Existing evaluations typically focus on explicit goal achievement or direct norm knowledge, seldom assessing whether planners can infer and apply these hidden constraints within action sequences. We introduce NormAct, a benchmark for embodied social-norm interactions that evaluates plans on Goal Achievement, Norm Compliance, and overall Task Success. NormAct uniquely embeds hidden norms within ordinary tasks, testing whether models can realize them without explicit instruction. Experiments with state-of-the-art MLLMs (GPT-5.4, Claude Opus 4.7, Gemini 3 Pro) reveal a significant gap: models achieve explicit goals in 67.3\% of cases, but comply with hidden norms in only 26.4\%. Cue-condition experiments indicate that this gap stems not from a lack of general social knowledge, but from challenges in activating and grounding relevant norms in context. To address this, we propose NormPerceptor, a context-conditioned cue generator that infers scene-relevant norms prior to planning, increasing Task Success from 24.2\% to 46.7\%. Our results underscore the importance of enabling embodied agents to proactively detect hidden norms, ground them in visual evidence, and integrate them as action-planning constraints. Our benchmark is publicly available at https://huggingface.co/datasets/Caleb196x/NormAct.

Related papers

• xDailyBench: Benchmarking LLMs on Professional Consultation for Real-Life Problems [57.62172039072062]
  We introduce xDailyBench, a benchmark of 248 carefully curated tasks spanning 51 scenarios across personal life, white-collar work, learning and research, and cross-domain activities.<n>The tasks are grounded in requests that users have actually completed or genuinely intended to accomplish with AI, and are evaluated with fine-grained binary rubrics covering both explicit and implicit requirements.<n>The best models achieve a task-level score of 75.6%, while all models perform substantially worse on implicit than explicit requirements, with gaps no less than 9 percentage points.
  arXiv  Detail & Related papers  (2026-09-07T17:30:58Z)
• Grounded Normative Rule Generation with Structured Search [47.28595196084396]
  We formalize the problem as Grounded Normative Rule Synthesis (GNRS)<n>We introduce GNRS-Search, a framework that utilizes Markov Chain Monte Carlo (MCMC) sampling to optimize a discrete, five-slot And-Or Graph (AOG)<n>We evaluate our approach on GNRS-Bench, a benchmark spanning 116 controlled goals across eight scene families, and RealCharterBench, which evaluates transfer to 53 real-derived policy tasks with hidden source clauses.
  arXiv  Detail & Related papers  (2026-08-23T05:43:42Z)
• GAIN: A Benchmark for Goal-Aligned Decision-Making of Large Language Models under Imperfect Norms [3.6158033114580683]
  We introduce a benchmark designed to evaluate how large language models balance adherence to norms against business goals.<n>We define five types of pressures: Goal Alignment, Risk Aversion, Emotional/Appeal, Social/Authoritative Influence, and Personal Incentive.<n>The benchmark comprises 1,200 scenarios across four domains: hiring, customer support, advertising and finance.
  arXiv  Detail & Related papers  (2026-03-19T04:04:45Z)
• Where Norms and References Collide: Evaluating LLMs on Normative Reasoning [3.8431932182760296]
  Embodied agents, such as robots, will need to interact in situated environments where successful communication often depends on reasoning over social norms.<n>It remains unclear whether Large Language Models (LLMs) can support this kind of reasoning.<n>We introduce SNIC (Situated Norms in Context), a human-validated diagnostic testbed designed to probe how well state-of-the-art LLMs can extract and utilize normative principles relevant to NBRR.
  arXiv  Detail & Related papers  (2026-02-03T01:23:22Z)
• Beyond Log Likelihood: Probability-Based Objectives for Supervised Fine-Tuning across the Model Capability Continuum [88.90314335542281]
  We study a family of probability-based objectives and characterize their effectiveness under different conditions.<n>We uncover a critical dimension that governs objective behavior: the model-capability continuum.<n>Our theoretical analysis further elucidates how objectives trade places across the continuum, providing a principled foundation for adapting objectives to model capability.
  arXiv  Detail & Related papers  (2025-10-01T05:17:47Z)
• Measuring Physical-World Privacy Awareness of Large Language Models: An Evaluation Benchmark [10.645183619391135]
  We introduce EAPrivacy, a comprehensive evaluation benchmark designed to quantify the physical-world privacy awareness of embodied agents.<n>EAPrivacy utilizes procedurally generated scenarios across four tiers to test an agent's ability to handle sensitive objects, adapt to changing environments, balance task execution with privacy constraints, and resolve conflicts with social norms.
  arXiv  Detail & Related papers  (2025-09-27T23:39:56Z)
• SConU: Selective Conformal Uncertainty in Large Language Models [59.25881667640868]
  We propose a novel approach termed Selective Conformal Uncertainty (SConU)<n>We develop two conformal p-values that are instrumental in determining whether a given sample deviates from the uncertainty distribution of the calibration set at a specific manageable risk level.<n>Our approach not only facilitates rigorous management of miscoverage rates across both single-domain and interdisciplinary contexts, but also enhances the efficiency of predictions.
  arXiv  Detail & Related papers  (2025-04-19T03:01:45Z)
• EgoNormia: Benchmarking Physical Social Norm Understanding [52.87904722234434]
  EGONORMIA spans seven norm categories: safety, privacy, proxemics, politeness, cooperation, coordination/proactivity, and communication/legibility.<n>Our work demonstrates that current state-of-the-art vision-language models (VLMs) lack robust grounded norm understanding, scoring a maximum of 54% on EGONORMIA and 65% on EGONORMIA-verified.
  arXiv  Detail & Related papers  (2025-02-27T19:54:16Z)
• PredictaBoard: Benchmarking LLM Score Predictability [50.47497036981544]
  Large Language Models (LLMs) often fail unpredictably.<n>This poses a significant challenge to ensuring their safe deployment.<n>We present PredictaBoard, a novel collaborative benchmarking framework.
  arXiv  Detail & Related papers  (2025-02-20T10:52:38Z)
• Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences [36.884156839960184]
  We investigate whether contemporary NLG models can function as behavioral priors for systems deployed in social settings. We introduce 'Moral Stories', a crowd-sourced dataset of structured, branching narratives for the study of grounded, goal-oriented social reasoning.
  arXiv  Detail & Related papers  (2020-12-31T17:28:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.