NARU: A Benchmark for NARrative Evolution and Cultural Nuance Understanding in Japanese Extreme Long Video

• URL: http://arxiv.org/abs/2608.13210v1
• Date: Thu, 13 Aug 2026 13:13:43 GMT
• Title: NARU: A Benchmark for NARrative Evolution and Cultural Nuance Understanding in Japanese Extreme Long Video
• Abstract summary: We introduce NARU, a benchmark designed to evaluate Narrative evolution and Reasoning on cultural understanding in Japanese long-form video.<n> NARU consists of 1,481 questions grounded in 155 videos totaling 146.8 hours, spanning four narrative and five cultural dimensions.
• Score: 14.8531001537093
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-form video understanding encompasses tasks that go beyond retrieving isolated events, including tracking an evolving narrative and interpreting social meaning that may remain implicit. However, existing benchmarks rarely evaluate these capabilities jointly, particularly in high-context, non-English media. To address this gap, we introduce NARU, a benchmark designed to evaluate Narrative evolution and Reasoning on cultural Understanding in Japanese long-form video. NARU consists of 1,481 questions grounded in 155 videos totaling 146.8 hours, spanning four narrative and five cultural dimensions. To construct the benchmark at this scale, we propose a hierarchical memory-based annotation pipeline that transforms raw video into structured event, narrative, and cultural annotations, then generates questions via task-oriented synthesis and iterative shortcut removal. The construction process includes two native-speaker verification stages involving 68 annotators. Evaluations across eight model configurations reveal substantial limitations in both long-range narrative integration and culturally grounded reasoning. By exposing these persistent gaps, NARU offers a systematic testing ground for developing MLLMs capable of reliably interpreting long-form, high-context video.

Related papers

• NEST: Narrative Event Structures in Time for Long Video Understanding [6.938433277109506]
  We introduce NEST (Narrative Event Structures in Time for Long Video Understanding), a dataset of 1005 full-length movies (avg. 98 minutes)<n>NEST captures multimodal narrative events with structured annotations grounded in visual content, dialogue, and audio, and links them through relations that reflect narrative structure.<n>The benchmark is highly challenging for grounded event discovery, with ETD below 8%, EL under 6%, and EAE below 11%.
  arXiv  Detail & Related papers  (2026-06-18T02:05:14Z)
• Narrative Aligned Long Form Video Question Answering [12.805359127629215]
  We introduce NA-VQA, a benchmark designed to evaluate deep temporal and narrative reasoning in long-form videos.<n> NA-VQA contains 88 full-length movies and 4.4K open-ended question-answer pairs, each grounded in multiple evidence spans labeled as Short, Medium, or Far.<n>We propose Video-NaRA, a narrative-centric framework that builds event-level chains and stores them in a structured memory for retrieval during reasoning.
  arXiv  Detail & Related papers  (2026-03-19T21:23:15Z)
• CURVE: A Benchmark for Cultural and Multilingual Long Video Reasoning [58.73855961335903]
  CURVE (Cultural Understanding and Reasoning in Video Evaluation) is a challenging benchmark for multicultural and multilingual video reasoning.<n>It comprises high-quality, entirely human-generated annotations from diverse, region-specific cultural videos across 18 global locales.<n>Our evaluations reveal that SoTA Video-LLMs struggle significantly, performing substantially below human-level accuracy.
  arXiv  Detail & Related papers  (2026-01-15T18:15:06Z)
• Cinéaste: A Fine-grained Contextual Movie Question Answering Benchmark [47.482960367243756]
  We introduce $mathsfCinacuteeaste$, a comprehensive benchmark for long-form movie understanding.<n>Our dataset comprises 3,119 multiple-choice question-answer pairs derived from 1,805 scenes across 200 movies.<n>Experiments show that existing MLLMs struggle on $mathsfCinacuteeaste$; our analysis reveals that long-range temporal reasoning is a primary bottleneck.
  arXiv  Detail & Related papers  (2025-09-17T17:58:06Z)
• SAMA: Towards Multi-Turn Referential Grounded Video Chat with Large Language Models [93.73583158211115]
  Achieving fine-grained-temporal understanding in videos remains a major challenge for current Video Large Multimodels (Video LMMs)<n>We contribute in three core aspects: dataset, model, and benchmark.<n>First, we introduce SAMA-239K, a large-scale dataset comprising 15K videos specifically to enable joint learning of video understanding, grounding, and multi-turn video chat.<n>Second, we propose the SAMA model, which incorporates a versatile-temporal context aggregator and a Segment Model to jointly enhance fine-grained video comprehension and precise grounding capabilities.
  arXiv  Detail & Related papers  (2025-05-24T18:13:16Z)
• E.T. Bench: Towards Open-Ended Event-Level Video-Language Understanding [57.630136434038384]
  We introduce E.T. Bench (Event-Level & Time-Sensitive Video Understanding Benchmark), a large-scale benchmark for open-ended event-level video understanding. We extensively evaluated 8 Image-LLMs and 12 Video-LLMs on our benchmark, and the results reveal that state-of-the-art models for coarse-level (video-level) understanding struggle to solve our fine-grained tasks. Our simple but effective solution demonstrates superior performance in multiple scenarios.
  arXiv  Detail & Related papers  (2024-09-26T17:53:04Z)
• NovelQA: Benchmarking Question Answering on Documents Exceeding 200K Tokens [63.7488938083696]
  We introduce NovelQA, a benchmark tailored for evaluating Large Language Models (LLMs) with complex, extended narratives.<n>NovelQA offers a unique blend of complexity, length, and narrative coherence, making it an ideal tool for assessing deep textual understanding.<n>Our evaluation of long-context LLMs on NovelQA reveals significant insights into their strengths and weaknesses.
  arXiv  Detail & Related papers  (2024-03-18T17:32:32Z)
• Building Scalable Video Understanding Benchmarks through Sports [22.851883205547644]
  We introduce an automated. Video Stream Alignment Pipeline (abbreviated ASAP) We demonstrate the generality of ASAP by aligning unlabeled videos of four different sports with corresponding dense web annotations. We then leverage ASAP scalability to create LCric, a large-scale long video understanding benchmark.
  arXiv  Detail & Related papers  (2023-01-17T13:20:21Z)
• VIOLIN: A Large-Scale Dataset for Video-and-Language Inference [103.7457132841367]
  We introduce a new task, Video-and-Language Inference, for joint multimodal understanding of video and text. Given a video clip with subtitles aligned as premise, paired with a natural language hypothesis based on the video content, a model needs to infer whether the hypothesis is entailed or contradicted by the given video clip. A new large-scale dataset, named Violin (VIdeO-and-Language INference), is introduced for this task, which consists of 95,322 video-hypothesis pairs from 15,887 video clips.
  arXiv  Detail & Related papers  (2020-03-25T20:39:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.