A Text Classification-Based Approach for Evaluating and Enhancing the Machine Interpretability of Building Codes

• URL: http://arxiv.org/abs/2309.14374v1
• Date: Sun, 24 Sep 2023 11:36:21 GMT
• Title: A Text Classification-Based Approach for Evaluating and Enhancing the Machine Interpretability of Building Codes
• Authors: Zhe Zheng, Yu-Cheng Zhou, Ke-Yin Chen, Xin-Zheng Lu, Zhong-Tian She, Jia-Rui Lin
• Abstract summary: This research aims to propose a novel approach to automatically evaluate and enhance the machine interpretability of single clause and building codes. Experiments show that the proposed text classification algorithm outperforms the existing CNN- or RNN-based methods. analyzing the results of more than 150 building codes in China showed that their average interpretability is 34.40%.
• Score: 9.730183895717056
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Interpreting regulatory documents or building codes into computer-processable formats is essential for the intelligent design and construction of buildings and infrastructures. Although automated rule interpretation (ARI) methods have been investigated for years, most of them highly depend on the early and manual filtering of interpretable clauses from a building code. While few of them considered machine interpretability, which represents the potential to be transformed into a computer-processable format, from both clause- and document-level. Therefore, this research aims to propose a novel approach to automatically evaluate and enhance the machine interpretability of single clause and building codes. First, a few categories are introduced to classify each clause in a building code considering the requirements for rule interpretation, and a dataset is developed for model training. Then, an efficient text classification model is developed based on a pretrained domain-specific language model and transfer learning techniques. Finally, a quantitative evaluation method is proposed to assess the overall interpretability of building codes. Experiments show that the proposed text classification algorithm outperforms the existing CNN- or RNN-based methods, improving the F1-score from 72.16% to 93.60%. It is also illustrated that the proposed classification method can enhance downstream ARI methods with an improvement of 4%. Furthermore, analyzing the results of more than 150 building codes in China showed that their average interpretability is 34.40%, which implies that it is still hard to fully transform the entire regulatory document into computer-processable formats. It is also argued that the interpretability of building codes should be further improved both from the human side and the machine side.

Related papers

• Empirical Evaluation of Embedding Models in the Context of Text Classification in Document Review in Construction Delay Disputes [6.076874513889027]
  Text embeddings are numerical representations of text data, where words, phrases, or entire documents are converted into vectors of real numbers. This paper presents our work in evaluating different embeddings through a comprehensive comparative analysis of four distinct models. We employ both K-Nearest Neighbors (KNN) and Logistic Regression (LR) to perform binary classification tasks, specifically determining whether a text snippet is associated with 'delay' or 'not delay' within a labeled dataset.
  arXiv  Detail & Related papers  (2025-01-16T22:12:11Z)
• Are Large Language Models Good Classifiers? A Study on Edit Intent Classification in Scientific Document Revisions [62.12545440385489]
  Large language models (LLMs) have brought substantial advancements in text generation, but their potential for enhancing classification tasks remains underexplored. We propose a framework for thoroughly investigating fine-tuning LLMs for classification, including both generation- and encoding-based approaches. We instantiate this framework in edit intent classification (EIC), a challenging and underexplored classification task.
  arXiv  Detail & Related papers  (2024-10-02T20:48:28Z)
• Blueprinting the Future: Automatic Item Categorization using Hierarchical Zero-Shot and Few-Shot Classifiers [6.907552533477328]
  This study unveils a novel approach employing the zero-shot and few-shot Generative Pretrained Transformer (GPT) for hierarchical item categorization. The hierarchical nature of examination blueprints is navigated seamlessly, allowing for a tiered classification of items across multiple levels. An initial simulation with artificial data demonstrates the efficacy of this method, achieving an average accuracy of 92.91% measured by the F1 score.
  arXiv  Detail & Related papers  (2023-12-06T15:51:49Z)
• Scalable Learning of Latent Language Structure With Logical Offline Cycle Consistency [71.42261918225773]
  Conceptually, LOCCO can be viewed as a form of self-learning where the semantic being trained is used to generate annotations for unlabeled text. As an added bonus, the annotations produced by LOCCO can be trivially repurposed to train a neural text generation model.
  arXiv  Detail & Related papers  (2023-05-31T16:47:20Z)
• A Machine Learning Approach to Classifying Construction Cost Documents into the International Construction Measurement Standard [0.0]
  We introduce the first automated models for classifying natural language descriptions provided in cost documents called "Bills of Quantities" We learn from a dataset of more than 50 thousand descriptions of items retrieved from 24 large infrastructure construction projects across the United Kingdom.
  arXiv  Detail & Related papers  (2022-10-24T11:35:53Z)
• Computer-Assisted Creation of Boolean Search Rules for Text Classification in the Legal Domain [0.5249805590164901]
  We develop an interactive environment called CASE which exploits word co-occurrence to guide human annotators in selection of relevant search terms. The system seamlessly facilitates iterative evaluation and improvement of the classification rules. We evaluate classifiers created with our CASE system on 4 datasets, and compare the results to machine learning methods.
  arXiv  Detail & Related papers  (2021-12-10T19:53:41Z)
• LeQua@CLEF2022: Learning to Quantify [76.22817970624875]
  LeQua 2022 is a new lab for the evaluation of methods for learning to quantify'' in textual datasets. The goal of this lab is to provide a setting for the comparative evaluation of methods for learning to quantify, both in the binary setting and in the single-label multiclass setting.
  arXiv  Detail & Related papers  (2021-11-22T14:54:20Z)
• Data Augmentation in Natural Language Processing: A Novel Text Generation Approach for Long and Short Text Classifiers [8.19984844136462]
  We present and evaluate a text generation method suitable to increase the performance of classifiers for long and short texts. In a simulated low data regime additive accuracy gains of up to 15.53% are achieved. We discuss implications and patterns for the successful application of our approach on different types of datasets.
  arXiv  Detail & Related papers  (2021-03-26T13:16:07Z)
• TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing [73.16475763422446]
  We propose a multilingual robustness evaluation platform for NLP tasks (TextFlint) It incorporates universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analysis. TextFlint generates complete analytical reports as well as targeted augmented data to address the shortcomings of the model's robustness.
  arXiv  Detail & Related papers  (2021-03-21T17:20:38Z)
• Hierarchical Bi-Directional Self-Attention Networks for Paper Review Rating Recommendation [81.55533657694016]
  We propose a Hierarchical bi-directional self-attention Network framework (HabNet) for paper review rating prediction and recommendation. Specifically, we leverage the hierarchical structure of the paper reviews with three levels of encoders: sentence encoder (level one), intra-review encoder (level two) and inter-review encoder (level three) We are able to identify useful predictors to make the final acceptance decision, as well as to help discover the inconsistency between numerical review ratings and text sentiment conveyed by reviewers.
  arXiv  Detail & Related papers  (2020-11-02T08:07:50Z)
• A Diagnostic Study of Explainability Techniques for Text Classification [52.879658637466605]
  We develop a list of diagnostic properties for evaluating existing explainability techniques. We compare the saliency scores assigned by the explainability techniques with human annotations of salient input regions to find relations between a model's performance and the agreement of its rationales with human ones.
  arXiv  Detail & Related papers  (2020-09-25T12:01:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.