INS-ActBench: A Comprehensive Benchmark for Assessing Professional Actuarial Capability of Large Language Models

• URL: http://arxiv.org/abs/2607.24273v1
• Date: Mon, 27 Jul 2026 11:12:50 GMT
• Title: INS-ActBench: A Comprehensive Benchmark for Assessing Professional Actuarial Capability of Large Language Models
• Abstract summary: INS-ActBench is a benchmark for evaluating actuarial capability in large language models (LLMs)<n> INS-ActBench contains 12,050 Q&A pairs from public exams and sample questions released by 16 actuarial associations.<n>It covers three subsets: textbfINS-Act-Know for standardized actuarial knowledge, textbfINS-Act-Case for long-context insurance case reasoning, and textbfINS-Act-Practice for spreadsheet and R-code tasks with numerical outputs.
• Score: 11.556217835954842
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) have shown strong potential in financial reasoning, but existing benchmarks often evaluate domain knowledge, numerical reasoning, long-context understanding, and tool use in separate settings. This limits their ability to assess realistic professional workflows that require auditable, context-grounded, and tool-executable decisions. We introduce \textbf{INS-ActBench}, a comprehensive benchmark for evaluating professional actuarial capability in LLMs. INS-ActBench contains 12,050 Q\&A pairs from public exams and sample questions released by 16 actuarial associations. It covers three subsets: \textbf{INS-Act-Know} for standardized actuarial knowledge, \textbf{INS-Act-Case} for long-context insurance case reasoning, and \textbf{INS-Act-Practice} for spreadsheet and R-code tasks with verifiable numerical outputs. Experiments on nine representative LLMs and human actuarial experts reveal a clear capability boundary: frontier LLMs perform strongly on standardized knowledge, but remain much weaker in case reasoning, tool-based workflows, and jurisdiction-sensitive practice. INS-ActBench provides a reproducible foundation for developing actuarial LLMs toward reliable professional assistance. The code is available at https://github.com/FDU-INS/INS-ActBench.

Related papers

• Beyond the Syntax: Do Security Experts Trust LLMs for NIDS Rule Engineering? [70.17374586573033]
  This paper presents a human-centered investigation into Large Language Models (LLMs)-based NIDS rule engineering.<n>Our evaluation reveals a syntax-semantics paradox: although LLMs generate syntactically correct rules, experts find them only partially deployable.<n>While large-scale models consistently produce syntactically valid rules, small models are largely ineffective for IDS rule generation.
  arXiv  Detail & Related papers  (2026-07-07T07:09:56Z)
• \$OneMillion-Bench: How Far are Language Agents from Human Experts? [43.31541643833527]
  $OneMillion-Bench is a benchmark of 400 expert-curated tasks spanning Law, Finance, Industry, Healthcare, and Natural Science.<n>$OneMillion-Bench provides a unified testbed for assessing agentic reliability, professional depth, and practical readiness in domain-intensive scenarios.
  arXiv  Detail & Related papers  (2026-03-09T05:32:42Z)
• PLawBench: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practice [67.71760070255425]
  We introduce PLawBench, a practical benchmark for evaluating large language models (LLMs) in legal practice scenarios.<n>PLawBench comprises 850 questions across 13 practical legal scenarios, with each question accompanied by expert-designed evaluation rubrics.<n>Using an LLM-based evaluator aligned with human expert judgments, we evaluate 10 state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2026-01-23T11:36:10Z)
• AutoBaxBuilder: Bootstrapping Code Security Benchmarking [14.946765026951601]
  We present AutoBaxBuilder, a framework that generates tasks and tests for code security benchmarking from scratch.<n>We introduce a robust pipeline with fine-grained plausibility checks, leveraging the code understanding capabilities of LLMs to construct functionality tests and end-to-end security-probing exploits.<n>We use AutoBaxBuilder to construct entirely new tasks and release them to the public as AutoBaxBench, together with a thorough evaluation of the security capabilities of LLMs on these tasks.
  arXiv  Detail & Related papers  (2025-12-24T12:02:00Z)
• TRUEBench: Can LLM Response Meet Real-world Constraints as Productivity Assistant? [11.400738388392654]
  Large language models (LLMs) are increasingly integral as productivity assistants.<n>Existing benchmarks fall short in rigorously evaluating their real-world instruction-following capabilities.<n>We introduce TRUEBench, a benchmark specifically designed for LLM-based productivity assistants.
  arXiv  Detail & Related papers  (2025-09-24T08:05:32Z)
• IDA-Bench: Evaluating LLMs on Interactive Guided Data Analysis [60.32962597618861]
  IDA-Bench is a novel benchmark evaluating large language models in multi-round interactive scenarios.<n>Agent performance is judged by comparing its final numerical output to the human-derived baseline.<n>Even state-of-the-art coding agents (like Claude-3.7-thinking) succeed on 50% of the tasks, highlighting limitations not evident in single-turn tests.
  arXiv  Detail & Related papers  (2025-05-23T09:37:52Z)
• Tests as Prompt: A Test-Driven-Development Benchmark for LLM Code Generation [1.7268889851975326]
  We introduce WebApp1K, a novel benchmark for evaluating large language models (LLMs) in test-driven development (TDD) tasks.<n>Unlike traditional approaches relying on natural language prompts, our benchmark emphasizes the ability of LLMs to interpret and implement functionality directly from test cases.
  arXiv  Detail & Related papers  (2025-05-13T23:47:12Z)
• Computational Reasoning of Large Language Models [51.629694188014064]
  We introduce textbfTuring Machine Bench, a benchmark to assess the ability of Large Language Models (LLMs) to execute reasoning processes.<n> TMBench incorporates four key features: self-contained and knowledge-agnostic reasoning, a minimalistic multi-step structure, controllable difficulty, and a theoretical foundation based on Turing machine.
  arXiv  Detail & Related papers  (2025-04-29T13:52:47Z)
• CodeIF-Bench: Evaluating Instruction-Following Capabilities of Large Language Models in Interactive Code Generation [10.438717413104062]
  We introduce CodeIF-Bench, a benchmark for evaluating the instruction-following capabilities of Large Language Models.<n>CodeIF-Bench incorporates nine types of verifiable instructions aligned with the real-world software development requirements.<n>In both textitStatic Conversation and textitDynamic Conversation settings, we evaluate the performance of 7 state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2025-03-05T09:47:02Z)
• Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More? [54.667202878390526]
  Long-context language models (LCLMs) have the potential to revolutionize our approach to tasks traditionally reliant on external tools like retrieval systems or databases. We introduce LOFT, a benchmark of real-world tasks requiring context up to millions of tokens designed to evaluate LCLMs' performance on in-context retrieval and reasoning. Our findings reveal LCLMs' surprising ability to rival state-of-the-art retrieval and RAG systems, despite never having been explicitly trained for these tasks.
  arXiv  Detail & Related papers  (2024-06-19T00:28:58Z)
• INS-MMBench: A Comprehensive Benchmark for Evaluating LVLMs' Performance in Insurance [51.36387171207314]
  Large Vision-Language Models (LVLMs) and Multimodal Large Language Models (MLLMs) have shown increasing promise in specialized domains.<n>This study systematically reviews and categorizes multimodal tasks for 4 representative types of insurance: auto, property, health, and agricultural.<n>We benchmark 11 leading LVLMs, including closed-source models such as GPT-4o and open-source models like LLaVA.
  arXiv  Detail & Related papers  (2024-06-13T13:31:49Z)
• CoIN: A Benchmark of Continual Instruction tuNing for Multimodel Large Language Model [121.23360004498893]
  We present a benchmark, namely Continual Instruction tuNing (CoIN), to assess existing MLLMs in the sequential instruction tuning paradigm. Experiments on CoIN demonstrate that current powerful MLLMs still suffer catastrophic forgetting. We introduce MoELoRA to MLLMs which is effective to retain the previous instruction alignment.
  arXiv  Detail & Related papers  (2024-03-13T08:54:31Z)
• Through the Lens of Core Competency: Survey on Evaluation of Large Language Models [27.271533306818732]
  Large language model (LLM) has excellent performance and wide practical uses. Existing evaluation tasks are difficult to keep up with the wide range of applications in real-world scenarios. We summarize 4 core competencies of LLM, including reasoning, knowledge, reliability, and safety. Under this competency architecture, similar tasks are combined to reflect corresponding ability, while new tasks can also be easily added into the system.
  arXiv  Detail & Related papers  (2023-08-15T17:40:34Z)
• CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Models [74.22729793816451]
  Large Language Models (LLMs) have made significant progress in utilizing tools, but their ability is limited by API availability. We propose CREATOR, a novel framework that enables LLMs to create their own tools using documentation and code realization. We evaluate CREATOR on MATH and TabMWP benchmarks, respectively consisting of challenging math competition problems.
  arXiv  Detail & Related papers  (2023-05-23T17:51:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.