From Interpretability to Control: Insights from Six Years of the TrustNLP Workshop

• URL: http://arxiv.org/abs/2608.11171v1
• Date: Tue, 11 Aug 2026 17:30:16 GMT
• Title: From Interpretability to Control: Insights from Six Years of the TrustNLP Workshop
• Abstract summary: The Workshop on Trustworthy Natural Language Processing has grown from 8 proceedings papers to 41 over six editions.<n>We synthesize insights from all 144 proceedings papers, classifying them along six trust dimensions grounded in established frameworks.<n>Analysis from the classification study reveals that truthfulness is the fastest-growing dimension (absent in 2021-2022, comprising 37% of papers by 2025-2026)
• Score: 63.00693129203923
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Workshop on Trustworthy Natural Language Processing (TrustNLP), co-located with major ACL conferences since 2021, has grown from 8 proceedings papers to 41 over six editions, documenting a field-wide transition from post-hoc interpretability of static models to mechanistic understanding and proactive control of generative systems. We synthesize insights from all 144 proceedings papers, classifying them along six trust dimensions grounded in established frameworks (TrustLLM, DecodingTrust). We observe co-occurrences with capability emergence. The release of the first high-impact chat models activated all trust dimensions simultaneously, while subsequent model generations shifted focus toward truthfulness and safety alignment. Analysis from the classification study reveals that truthfulness is the fastest-growing dimension (absent in 2021-2022, comprising 37% of papers by 2025-2026), fairness remains the most consistent theme, and explainability exhibits a U-shaped trajectory; declining as post-hoc methods lost relevance but resurging in 2026 through mechanistic interpretability. A cross-venue comparison with ACL, NAACL, EACL, and EMNLP (~2K papers) in the same period shows that TrustNLP's topical distribution closely follows the field average. We identify four structural insights and conclude with actionable directions for the research community.

Related papers

• FormalTCS: Benchmarking End-to-End Frontier Formal Theoretical Computer Science Research of Large Language Models [53.68149869349268]
  Large language models (LLMs) have shown growing potential for automated theoretical computer science (TCS) research.<n>We introduce ourbenchmark, an expert-validated benchmark for evaluating LLMs on frontier, end-to-end TCS research.
  arXiv  Detail & Related papers  (2026-08-20T15:13:41Z)
• Large Multimodal Agents for Intelligent Transportation Systems: Architectures, Evidence, and Deployment Challenges [10.296219658873676]
  This review provides an auditable evidence map of 42 primary study families released between January 2023 and 3 August 2026.<n>It distinguishes model-level, system-level, and hybrid multimodality and classifies each family by system architecture and action authority.
  arXiv  Detail & Related papers  (2026-08-08T15:22:40Z)
• From Isolated Tasks to Structured Capabilities: A Multilayer Taxonomy for Large Language Models [41.36461541603076]
  Large language model (LLM) evaluation spans diverse tasks and benchmarks.<n>Evidence remains organized around tasks rather than the capabilities they probe.<n>We introduce a multi-layer taxonomy of 14 capability domains and 91 subskills.
  arXiv  Detail & Related papers  (2026-07-24T10:49:50Z)
• Theory-Scale Auto-Formalization of Logics for Computer Science [36.45195518081978]
  LCS-Bench is a theory-scale benchmark based on Logics for Computer Science.<n>The artifact covers 327 textbook items, over 4,076 Lean declarations, and more than 85K lines of Lean code.<n>We demonstrate that LCS-Bench is of high quality, consistent, and faithful.
  arXiv  Detail & Related papers  (2026-06-25T01:59:53Z)
• Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models? [0.0]
  Chain-of-thought (CoT) reasoning has been proposed as a transparency mechanism for large language models in safety-critical deployments.<n>This study tests 12 open-weight reasoning models spanning 9 architectural families on 498 multiple-choice questions.<n>Across 41,832 inference runs, overall faithfulness rates range from 39.7% (Seed-1.6-Flash) to 89.9% (DeepSeek-V3.2-Speciale) across model families.
  arXiv  Detail & Related papers  (2026-03-23T21:21:37Z)
• Multi-Axis Trust Modeling for Interpretable Account Hijacking Detection [1.0152838128195467]
  This paper proposes a Hadith-inspired multi-axis trust modeling framework.<n>We translate five trust axes - long-term integrity (adalah), behavioral precision (dabt), contextual continuity (isnad), cumulative reputation, and anomaly evidence - into a compact set of 26 semantically meaningful behavioral features for user accounts.<n>We evaluate the framework on the CLUE-LDS cloud activity dataset with injected account hijacking scenarios.
  arXiv  Detail & Related papers  (2026-02-20T19:36:30Z)
• CLUE: Non-parametric Verification from Experience via Hidden-State Clustering [64.50919789875233]
  We show that correctness of a solution is encoded as a geometrically separable signature within the trajectory of hidden activations.<n>ClUE consistently outperforms LLM-as-a-judge baselines and matches or exceeds modern confidence-based methods in reranking candidates.
  arXiv  Detail & Related papers  (2025-10-02T02:14:33Z)
• Mapping Trustworthiness in Large Language Models: A Bibliometric Analysis Bridging Theory to Practice [11.251046343609078]
  This study aims to bridge the gap between theoretical discussion and practical implementation by analysing research trends, definitions of trustworthiness, and practical techniques.<n>We identified 18 different definitions of trust/trustworthiness, with transparency, explainability and reliability emerging as the most common dimensions.<n>We identified 20 strategies to enhance LLM trustworthiness, with fine-tuning and retrieval-augmented generation (RAG) being the most prominent.
  arXiv  Detail & Related papers  (2025-02-27T14:02:33Z)
• Law of the Weakest Link: Cross Capabilities of Large Language Models [102.91861246827797]
  We show that Large Language Models (LLMs) exhibit the "Law of the Weakest Link," where cross-capability performance is significantly constrained by the weakest component. These results highlight the under-performance of LLMs in cross-capability tasks.
  arXiv  Detail & Related papers  (2024-09-30T05:12:01Z)
• DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Data [65.5290035371111]
  We introduce an approach to generate extensive Lean 4 proof data derived from high-school and undergraduate-level mathematical competition problems. We fine-tune the DeepSeekMath 7B model on this synthetic dataset, which comprises 8 million formal statements with proofs. Our model successfully proved 5 out of 148 problems in the Lean 4 Formalized International Mathematical Olympiad (FIMO) benchmark, while GPT-4 failed to prove any.
  arXiv  Detail & Related papers  (2024-05-23T09:03:42Z)
• Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews [51.453135368388686]
  We present an approach for estimating the fraction of text in a large corpus which is likely to be substantially modified or produced by a large language model (LLM) Our maximum likelihood model leverages expert-written and AI-generated reference texts to accurately and efficiently examine real-world LLM-use at the corpus level.
  arXiv  Detail & Related papers  (2024-03-11T21:51:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.