LILaC: Late Interacting in Layered Component Graph for Open-domain Multimodal Multihop Retrieval

• URL: http://arxiv.org/abs/2602.04263v1
• Date: Wed, 04 Feb 2026 06:55:48 GMT
• Title: LILaC: Late Interacting in Layered Component Graph for Open-domain Multimodal Multihop Retrieval
• Authors: Joohyung Yun, Doyup Lee, Wook-Shin Han,
• Abstract summary: LILaC is a multimodal retrieval framework featuring two core innovations.<n>First, we introduce a layered component graph, explicitly representing multimodal information at two layers.<n>Second, we develop a late-interaction-based subgraph retrieval method.
• Score: 13.855117422052315
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal document retrieval aims to retrieve query-relevant components from documents composed of textual, tabular, and visual elements. An effective multimodal retriever needs to handle two main challenges: (1) mitigate the effect of irrelevant contents caused by fixed, single-granular retrieval units, and (2) support multihop reasoning by effectively capturing semantic relationships among components within and across documents. To address these challenges, we propose LILaC, a multimodal retrieval framework featuring two core innovations. First, we introduce a layered component graph, explicitly representing multimodal information at two layers - each representing coarse and fine granularity - facilitating efficient yet precise reasoning. Second, we develop a late-interaction-based subgraph retrieval method, an edge-based approach that initially identifies coarse-grained nodes for efficient candidate generation, then performs fine-grained reasoning via late interaction. Extensive experiments demonstrate that LILaC achieves state-of-the-art retrieval performance on all five benchmarks, notably without additional fine-tuning. We make the artifacts publicly available at github.com/joohyung00/lilac.

Related papers

• Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understanding [49.26132236798123]
  Vision Language Models (VLMs) have gradually become a primary approach in document understanding.<n>We propose SLEUTH, a multi agent framework that orchestrates a retriever and four collaborative agents in a coarse to fine process.<n>The framework identifies key textual and visual clues within the retrieved pages, filters for salient visual evidence such as tables and charts, and analyzes the query to devise a reasoning strategy.
  arXiv  Detail & Related papers  (2025-11-28T03:09:40Z)
• URaG: Unified Retrieval and Generation in Multimodal LLMs for Efficient Long Document Understanding [55.45331924836242]
  We present URaG, a framework that Unifies Retrieval and Generation within a single MLLM.<n>We show that URaG achieves state-of-the-art performance while reducing computational overhead by 44-56%.
  arXiv  Detail & Related papers  (2025-11-13T17:54:09Z)
• Doc2Query++: Topic-Coverage based Document Expansion and its Application to Dense Retrieval via Dual-Index Fusion [8.523351031498839]
  Document expansion (DE) via query generation tackles vocabulary mismatch in sparse retrieval, yet faces limitations.<n>We introduce Doc2Query++, a DE framework that structures query generation by first inferring a document's latent topics.<n>We propose Dual-Index Fusion strategy that isolates text and query signals, boosting performance in dense settings.
  arXiv  Detail & Related papers  (2025-10-10T17:07:48Z)
• Recurrence Meets Transformers for Universal Multimodal Retrieval [59.92546492752452]
  ReT-2 is a unified retrieval model that supports multimodal queries composed of both images and text.<n>We evaluate ReT-2 on the challenging M2KR and M-BEIR benchmarks across different retrieval configurations.<n>When integrated into retrieval-augmented generation pipelines, ReT-2 also improves downstream performance on Encyclopedic-VQA and InfoSeek datasets.
  arXiv  Detail & Related papers  (2025-09-10T18:00:29Z)
• Hierarchical Lexical Graph for Enhanced Multi-Hop Retrieval [22.33550491040999]
  RAG grounds large language models in external evidence, yet it still falters when answers must be pieced together across semantically distant documents.<n>We build two plug-and-play retrievers: StatementGraphRAG and TopicGraphRAG.<n>Our methods outperform naive chunk-based RAG achieving an average relative improvement of 23.1% in retrieval recall and correctness.
  arXiv  Detail & Related papers  (2025-06-09T17:58:35Z)
• Divide by Question, Conquer by Agent: SPLIT-RAG with Question-Driven Graph Partitioning [62.640169289390535]
  SPLIT-RAG is a multi-agent RAG framework that addresses the limitations with question-driven semantic graph partitioning and collaborative subgraph retrieval.<n>The innovative framework first create Semantic Partitioning of Linked Information, then use the Type-Specialized knowledge base to achieve Multi-Agent RAG.<n>The attribute-aware graph segmentation manages to divide knowledge graphs into semantically coherent subgraphs, ensuring subgraphs align with different query types.<n>A hierarchical merging module resolves inconsistencies across subgraph-derived answers through logical verifications.
  arXiv  Detail & Related papers  (2025-05-20T06:44:34Z)
• Towards Text-Image Interleaved Retrieval [49.96332254241075]
  We introduce the text-image interleaved retrieval (TIIR) task, where the query and document are interleaved text-image sequences.<n>We construct a TIIR benchmark based on naturally interleaved wikiHow tutorials, where a specific pipeline is designed to generate interleaved queries.<n>We propose a novel Matryoshka Multimodal Embedder (MME), which compresses the number of visual tokens at different granularity.
  arXiv  Detail & Related papers  (2025-02-18T12:00:47Z)
• Leveraging Inter-Chunk Interactions for Enhanced Retrieval in Large Language Model-Based Question Answering [12.60063463163226]
  IIER captures the internal connections between document chunks by considering three types of interactions: structural, keyword, and semantic. It identifies multiple seed nodes based on the target question and iteratively searches for relevant chunks to gather supporting evidence. It refines the context and reasoning chain, aiding the large language model in reasoning and answer generation.
  arXiv  Detail & Related papers  (2024-08-06T02:39:55Z)
• CART: A Generative Cross-Modal Retrieval Framework with Coarse-To-Fine Semantic Modeling [53.97609687516371]
  Cross-modal retrieval aims to search for instances, which are semantically related to the query through the interaction of different modal data.<n>Traditional solutions utilize a single-tower or dual-tower framework to explicitly compute the score between queries and candidates.<n>We propose a generative cross-modal retrieval framework (CART) based on coarse-to-fine semantic modeling.
  arXiv  Detail & Related papers  (2024-06-25T12:47:04Z)
• Multi-View Document Representation Learning for Open-Domain Dense Retrieval [87.11836738011007]
  This paper proposes a multi-view document representation learning framework. It aims to produce multi-view embeddings to represent documents and enforce them to align with different queries. Experiments show our method outperforms recent works and achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2022-03-16T03:36:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.