A Multi-model Approach for Video Data Retrieval in Autonomous Vehicle Development

• URL: http://arxiv.org/abs/2410.03580v1
• Date: Fri, 4 Oct 2024 16:38:27 GMT
• Title: A Multi-model Approach for Video Data Retrieval in Autonomous Vehicle Development
• Authors: Jesper Knapp, Klas Moberg, Yuchuan Jin, Simin Sun, Miroslaw Staron,
• Abstract summary: This paper presents and evaluates a pipeline that allows searching for specific scenarios in log collections using natural language descriptions instead ofsql. Our approach achieved a mean score of 3.3, demonstrating the potential of using a multi-model architecture. We also present an interface that can visualize the query process and visualize the results.
• Score: 1.584511776109105
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous driving software generates enormous amounts of data every second, which software development organizations save for future analysis and testing in the form of logs. However, given the vast size of this data, locating specific scenarios within a collection of vehicle logs can be challenging. Writing the correct SQL queries to find these scenarios requires engineers to have a strong background in SQL and the specific databases in question, further complicating the search process. This paper presents and evaluates a pipeline that allows searching for specific scenarios in log collections using natural language descriptions instead of SQL. The generated descriptions were evaluated by engineers working with vehicle logs at the Zenseact on a scale from 1 to 5. Our approach achieved a mean score of 3.3, demonstrating the potential of using a multi-model architecture to improve the software development workflow. We also present an interface that can visualize the query process and visualize the results.

Related papers

• Text-to-SQL for Enterprise Data Analytics [6.08835924526836]
  We present insights from building an internal bot that enables LinkedIn's product managers, engineers, and operations teams to self-serve data insights from a large, dynamic data lake.<n>Our approach features three components. First, we construct a knowledge graph that captures up-to-date semantics by indexing database metadata, historical query logs, wikis, and code.<n>Second, we build a Text-to-one clustering agent that retrieves and ranks context from the knowledge graph, writes a query, and automatically corrects hallucinations and syntax errors.
  arXiv  Detail & Related papers  (2025-07-18T21:39:17Z)
• An Empirical study on LLM-based Log Retrieval for Software Engineering Metadata Management [1.8570591025615453]
  This paper introduces a Large Language Model (LLM)-supported approach that combines signal log data with video recordings from test drives.<n>It provides quantifiable metrics to evaluate the reliability of query results.<n> Evaluation on an open industrial dataset demonstrates improved efficiency and reliability in scenario retrieval.
  arXiv  Detail & Related papers  (2025-06-13T10:40:23Z)
• FABLE: A Novel Data-Flow Analysis Benchmark on Procedural Text for Large Language Model Evaluation [5.866040886735852]
  We introduce FABLE, a benchmark designed to assess large language models' understanding of data flow using structured, procedural text.<n>We evaluate three types of LLMs: a reasoning-focused model (DeepSeek-R1 8B), a general-purpose model (LLaMA 3.1 8B), and a code-specific model (Granite Code 8B)<n>Results show that the reasoning model achieves higher accuracy, but at the cost of over 20 times slower inference compared to the other models.
  arXiv  Detail & Related papers  (2025-05-30T06:32:34Z)
• Chatting with Logs: An exploratory study on Finetuning LLMs for LogQL [0.781670921955053]
  Logging is a critical function in modern distributed applications, but the lack of standardization in log query languages and formats creates significant challenges. We propose a novel natural language interface to address these inconsistencies and aide log query generation.
  arXiv  Detail & Related papers  (2024-12-04T14:06:24Z)
• Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows? [73.81908518992161]
  We introduce Spider2-V, the first multimodal agent benchmark focusing on professional data science and engineering. Spider2-V features real-world tasks in authentic computer environments and incorporating 20 enterprise-level professional applications. These tasks evaluate the ability of a multimodal agent to perform data-related tasks by writing code and managing the GUI in enterprise data software systems.
  arXiv  Detail & Related papers  (2024-07-15T17:54:37Z)
• UQE: A Query Engine for Unstructured Databases [71.49289088592842]
  We investigate the potential of Large Language Models to enable unstructured data analytics. We propose a new Universal Query Engine (UQE) that directly interrogates and draws insights from unstructured data collections.
  arXiv  Detail & Related papers  (2024-06-23T06:58:55Z)
• IQLS: Framework for leveraging Metadata to enable Large Language Model based queries to complex, versatile Data [0.20482269513546458]
  The Intelligent Query and Learning System (IQLS) simplifies the process by allowing natural language use to simplify data retrieval. It maps structured data into a framework based on the available metadata and available data models. The IQLS enables the agent to fulfill tasks given by the user query through interfaces.
  arXiv  Detail & Related papers  (2024-05-04T13:44:05Z)
• Text2Analysis: A Benchmark of Table Question Answering with Advanced Data Analysis and Unclear Queries [67.0083902913112]
  We develop the Text2Analysis benchmark, incorporating advanced analysis tasks. We also develop five innovative and effective annotation methods. We evaluate five state-of-the-art models using three different metrics.
  arXiv  Detail & Related papers  (2023-12-21T08:50:41Z)
• A Large-Scale Evaluation for Log Parsing Techniques: How Far Are We? [42.56249610409624]
  We provide a new collection of annotated log datasets, denoted Loghub-2.0, which can better reflect the characteristics of log data in real-world software systems. We conduct a thorough re-evaluation of 15 state-of-the-art logs in a more rigorous and practical setting. Particularly, we introduce a new evaluation metric to mitigate the sensitivity of existing metrics to imbalanced data distributions.
  arXiv  Detail & Related papers  (2023-08-21T16:24:15Z)
• Log Parsing Evaluation in the Era of Modern Software Systems [47.370291246632114]
  We focus on one integral part of automated log analysis, log parsing, which is the prerequisite to deriving any insights from logs. Our investigation reveals problematic aspects within the log parsing field, particularly its inefficiency in handling heterogeneous real-world logs. We propose a tool, Logchimera, that enables estimating log parsing performance in industry contexts.
  arXiv  Detail & Related papers  (2023-08-17T14:19:22Z)
• Neural Graph Reasoning: Complex Logical Query Answering Meets Graph Databases [63.96793270418793]
  Complex logical query answering (CLQA) is a recently emerged task of graph machine learning. We introduce the concept of Neural Graph Database (NGDBs) NGDB consists of a Neural Graph Storage and a Neural Graph Engine.
  arXiv  Detail & Related papers  (2023-03-26T04:03:37Z)
• Self-Supervised Log Parsing [59.04636530383049]
  Large-scale software systems generate massive volumes of semi-structured log records. Existing approaches rely on log-specifics or manual rule extraction. We propose NuLog that utilizes a self-supervised learning model and formulates the parsing task as masked language modeling.
  arXiv  Detail & Related papers  (2020-03-17T19:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.