Trim My View: An LLM-Based Code Query System for Module Retrieval in Robotic Firmware

• URL: http://arxiv.org/abs/2503.03969v1
• Date: Wed, 05 Mar 2025 23:40:17 GMT
• Title: Trim My View: An LLM-Based Code Query System for Module Retrieval in Robotic Firmware
• Authors: Sima Arasteh, Pegah Jandaghi, Nicolaas Weideman, Dennis Perepech, Mukund Raghothaman, Christophe Hauser, Luis Garcia,
• Abstract summary: We show how a combination of binary decomposition techniques, decompilation passes, and LLM-powered function summarization can be used to build an economical engine.<n>We instantiated this technique with three underlying open-source LLMs and measured its effectiveness in identifying modules in robotics firmware.
• Score: 7.308605205194037
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The software compilation process has a tendency to obscure the original design of the system and makes it difficult both to identify individual components and discern their purpose simply by examining the resulting binary code. Although decompilation techniques attempt to recover higher-level source code from the machine code in question, they are not fully able to restore the semantics of the original functions. Furthermore, binaries are often stripped of metadata, and this makes it challenging to reverse engineer complex binary software. In this paper we show how a combination of binary decomposition techniques, decompilation passes, and LLM-powered function summarization can be used to build an economical engine to identify modules in stripped binaries and associate them with high-level natural language descriptions. We instantiated this technique with three underlying open-source LLMs -- CodeQwen, DeepSeek-Coder and CodeStral -- and measured its effectiveness in identifying modules in robotics firmware. This experimental evaluation involved 467 modules from four devices from the ArduPilot software suite, and showed that CodeStral, the best-performing backend LLM, achieves an average F1-score of 0.68 with an online running time of just a handful of seconds.

Related papers

• An Empirical Study on the Effectiveness of Large Language Models for Binary Code Understanding [50.17907898478795]
  This work proposes a benchmark to evaluate the effectiveness of Large Language Models (LLMs) in real-world reverse engineering scenarios. Our evaluations reveal that existing LLMs can understand binary code to a certain extent, thereby improving the efficiency of binary code analysis.
  arXiv  Detail & Related papers  (2025-04-30T17:02:06Z)
• SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution [56.9361004704428]
  Large Language Models (LLMs) have demonstrated remarkable proficiency across a variety of complex tasks.<n>SWE-Fixer is a novel open-source framework designed to effectively and efficiently resolve GitHub issues.<n>We assess our approach on the SWE-Bench Lite and Verified benchmarks, achieving state-of-the-art performance among open-source models.
  arXiv  Detail & Related papers  (2025-01-09T07:54:24Z)
• Towards Specification-Driven LLM-Based Generation of Embedded Automotive Software [0.4369550829556578]
  The paper studies how code generation by LLMs can be combined with formal verification to produce critical embedded software. The goal is to automatically generate industrial-quality code from specifications only.
  arXiv  Detail & Related papers  (2024-11-20T12:38:17Z)
• Uncovering LLM-Generated Code: A Zero-Shot Synthetic Code Detector via Code Rewriting [78.48355455324688]
  We propose a novel zero-shot synthetic code detector based on the similarity between the original code and its LLM-rewritten variants. Our results demonstrate a significant improvement over existing SOTA synthetic content detectors.
  arXiv  Detail & Related papers  (2024-05-25T08:57:28Z)
• How Far Have We Gone in Binary Code Understanding Using Large Language Models [51.527805834378974]
  We propose a benchmark to evaluate the effectiveness of Large Language Models (LLMs) in binary code understanding. Our evaluations reveal that existing LLMs can understand binary code to a certain extent, thereby improving the efficiency of binary code analysis.
  arXiv  Detail & Related papers  (2024-04-15T14:44:08Z)
• LLM4Decompile: Decompiling Binary Code with Large Language Models [10.346311290153398]
  Decompilation aims to convert binary code to high-level source code, but traditional tools like Ghidra often produce results difficult to read and execute. We propose LLM4Decompile, the first and largest open-source LLM series (1.3B to 33B) trained to decompile binary code. The resulting models significantly outperform GPT-4o and Ghidra on the HumanEval and ExeBench benchmarks by over 100% in terms of re-executability rate.
  arXiv  Detail & Related papers  (2024-03-08T13:10:59Z)
• SparseCoder: Identifier-Aware Sparse Transformer for File-Level Code Summarization [51.67317895094664]
  This paper studies file-level code summarization, which can assist programmers in understanding and maintaining large source code projects. We propose SparseCoder, an identifier-aware sparse transformer for effectively handling long code sequences.
  arXiv  Detail & Related papers  (2024-01-26T09:23:27Z)
• ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code [76.84199699772903]
  ML-Bench is a benchmark rooted in real-world programming applications that leverage existing code repositories to perform tasks. To evaluate both Large Language Models (LLMs) and AI agents, two setups are employed: ML-LLM-Bench for assessing LLMs' text-to-code conversion within a predefined deployment environment, and ML-Agent-Bench for testing autonomous agents in an end-to-end task execution within a Linux sandbox environment.
  arXiv  Detail & Related papers  (2023-11-16T12:03:21Z)
• Guess & Sketch: Language Model Guided Transpilation [59.02147255276078]
  Learned transpilation offers an alternative to manual re-writing and engineering efforts. Probabilistic neural language models (LMs) produce plausible outputs for every input, but do so at the cost of guaranteed correctness. Guess & Sketch extracts alignment and confidence information from features of the LM then passes it to a symbolic solver to resolve semantic equivalence.
  arXiv  Detail & Related papers  (2023-09-25T15:42:18Z)
• CodeTF: One-stop Transformer Library for State-of-the-art Code LLM [72.1638273937025]
  We present CodeTF, an open-source Transformer-based library for state-of-the-art Code LLMs and code intelligence. Our library supports a collection of pretrained Code LLM models and popular code benchmarks. We hope CodeTF is able to bridge the gap between machine learning/generative AI and software engineering.
  arXiv  Detail & Related papers  (2023-05-31T05:24:48Z)
• Improving type information inferred by decompilers with supervised machine learning [0.0]
  In software reverse engineering, decompilation is the process of recovering source code from binary files. We build different classification models capable of inferring the high-level type returned by functions. Our system is able to predict function return types with a 79.1% F1-measure, whereas the best decompiler obtains a 30% F1-measure.
  arXiv  Detail & Related papers  (2021-01-19T11:45:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.