LLM-FuncMapper: Function Identification for Interpreting Complex Clauses
in Building Codes via LLM
- URL: http://arxiv.org/abs/2308.08728v1
- Date: Thu, 17 Aug 2023 01:58:04 GMT
- Title: LLM-FuncMapper: Function Identification for Interpreting Complex Clauses
in Building Codes via LLM
- Authors: Zhe Zheng, Ke-Yin Chen, Xin-Yu Cao, Xin-Zheng Lu, Jia-Rui Lin
- Abstract summary: LLM-FuncMapper is an approach to identifying predefined functions needed to interpret various regulatory clauses.
Almost 100% of computer-processible clauses can be interpreted and represented as computer-executable codes.
This study is the first attempt to introduce LLM for understanding and interpreting complex regulatory clauses.
- Score: 3.802984168589694
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: As a vital stage of automated rule checking (ARC), rule interpretation of
regulatory texts requires considerable effort. However, interpreting regulatory
clauses with implicit properties or complex computational logic is still
challenging due to the lack of domain knowledge and limited expressibility of
conventional logic representations. Thus, LLM-FuncMapper, an approach to
identifying predefined functions needed to interpret various regulatory clauses
based on the large language model (LLM), is proposed. First, by systematically
analysis of building codes, a series of atomic functions are defined to capture
shared computational logics of implicit properties and complex constraints,
creating a database of common blocks for interpreting regulatory clauses. Then,
a prompt template with the chain of thought is developed and further enhanced
with a classification-based tuning strategy, to enable common LLMs for
effective function identification. Finally, the proposed approach is validated
with statistical analysis, experiments, and proof of concept. Statistical
analysis reveals a long-tail distribution and high expressibility of the
developed function database, with which almost 100% of computer-processible
clauses can be interpreted and represented as computer-executable codes.
Experiments show that LLM-FuncMapper achieve promising results in identifying
relevant predefined functions for rule interpretation. Further proof of concept
in automated rule interpretation also demonstrates the possibility of
LLM-FuncMapper in interpreting complex regulatory clauses. To the best of our
knowledge, this study is the first attempt to introduce LLM for understanding
and interpreting complex regulatory clauses, which may shed light on further
adoption of LLM in the construction domain.
Related papers
- On the Design and Analysis of LLM-Based Algorithms [74.7126776018275]
Large language models (LLMs) are used as sub-routines in algorithms.
LLMs have achieved remarkable empirical success.
Our framework holds promise for advancing LLM-based algorithms.
To promote further study of LLM-based algorithms, we release our source code at https://github.com/modelscope/agentscope/tree/main/examples/paper_llm_based_algorithm.
arXiv Detail & Related papers (2024-07-20T07:39:07Z) - Large Language Models are Interpretable Learners [53.56735770834617]
In this paper, we show a combination of Large Language Models (LLMs) and symbolic programs can bridge the gap between expressiveness and interpretability.
The pretrained LLM with natural language prompts provides a massive set of interpretable modules that can transform raw input into natural language concepts.
As the knowledge learned by LSP is a combination of natural language descriptions and symbolic rules, it is easily transferable to humans (interpretable) and other LLMs.
arXiv Detail & Related papers (2024-06-25T02:18:15Z) - Guiding LLM Temporal Logic Generation with Explicit Separation of Data and Control [0.7580487359358722]
Temporal logics are powerful tools that are widely used for the synthesis and verification of reactive systems.
Recent progress on Large Language Models has the potential to make the process of writing such specifications more accessible.
arXiv Detail & Related papers (2024-06-11T16:07:24Z) - Potential and Limitations of LLMs in Capturing Structured Semantics: A Case Study on SRL [78.80673954827773]
Large Language Models (LLMs) play a crucial role in capturing structured semantics to enhance language understanding, improve interpretability, and reduce bias.
We propose using Semantic Role Labeling (SRL) as a fundamental task to explore LLMs' ability to extract structured semantics.
We find interesting potential: LLMs can indeed capture semantic structures, and scaling-up doesn't always mirror potential.
We are surprised to discover that significant overlap in the errors is made by both LLMs and untrained humans, accounting for almost 30% of all errors.
arXiv Detail & Related papers (2024-05-10T11:44:05Z) - Logic Agent: Enhancing Validity with Logic Rule Invocation [24.815341366820753]
Chain-of-Thought prompting has emerged as a pivotal technique for augmenting the inferential capabilities of language models during reasoning tasks.
This paper introduces the Logic Agent (LA), an agent-based framework aimed at enhancing the validity of reasoning processes in Large Language Models.
arXiv Detail & Related papers (2024-04-28T10:02:28Z) - $\forall$uto$\exists$val: Autonomous Assessment of LLMs in Formal Synthesis and Interpretation Tasks [21.12437562185667]
This paper presents a new approach for scaling LLM assessment in translating formal syntax to natural language.
We use context-free grammars (CFGs) to generate out-of-distribution datasets on the fly.
We also conduct an assessment of several SOTA closed and open-source LLMs to showcase the feasibility and scalability of this paradigm.
arXiv Detail & Related papers (2024-03-27T08:08:00Z) - Can LLMs Reason with Rules? Logic Scaffolding for Stress-Testing and Improving LLMs [87.34281749422756]
Large language models (LLMs) have achieved impressive human-like performance across various reasoning tasks.
However, their mastery of underlying inferential rules still falls short of human capabilities.
We propose a logic scaffolding inferential rule generation framework, to construct an inferential rule base, ULogic.
arXiv Detail & Related papers (2024-02-18T03:38:51Z) - If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code
Empowers Large Language Models to Serve as Intelligent Agents [81.60906807941188]
Large language models (LLMs) are trained on a combination of natural language and formal language (code)
Code translates high-level goals into executable steps, featuring standard syntax, logical consistency, abstraction, and modularity.
arXiv Detail & Related papers (2024-01-01T16:51:20Z) - Can Large Language Models Understand Real-World Complex Instructions? [54.86632921036983]
Large language models (LLMs) can understand human instructions, but struggle with complex instructions.
Existing benchmarks are insufficient to assess LLMs' ability to understand complex instructions.
We propose CELLO, a benchmark for evaluating LLMs' ability to follow complex instructions systematically.
arXiv Detail & Related papers (2023-09-17T04:18:39Z) - LMs: Understanding Code Syntax and Semantics for Code Analysis [25.508254718438636]
We evaluate the capabilities of large language models (LLMs) and their limitations for code analysis in software engineering.
We employ four state-of-the-art foundational models, GPT4, GPT3.5, StarCoder and CodeLlama-13b-instruct.
arXiv Detail & Related papers (2023-05-20T08:43:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.