IndustryCode: A Benchmark for Industry Code Generation

• URL: http://arxiv.org/abs/2604.02729v1
• Date: Fri, 03 Apr 2026 04:44:07 GMT
• Title: IndustryCode: A Benchmark for Industry Code Generation
• Abstract summary: We introduce IndustryCode, the first comprehensive benchmark designed to span multiple industrial domains and programming languages.<n>IndustryCode comprises 579 sub-problems derived from 125 primary industrial challenges, accompanied by rigorous problem descriptions and test cases.<n>In our evaluation, the top-performing model achieved an overall accuracy of 68.1% on sub-problems and 42.5% main problems.
• Score: 16.93701944012316
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Code generation and comprehension by Large Language Models (LLMs) have emerged as core drivers of industrial intelligence and decision optimization, finding widespread application in fields such as finance, automation, and aerospace. Although recent advancements have demonstrated the remarkable potential of LLMs in general code generation, existing benchmarks are mainly confined to single domains and languages. Consequently, they fail to effectively evaluate the generalization capabilities required for real-world industrial applications or to reflect the coding proficiency demanded by complex industrial scenarios. To bridge this gap, we introduce IndustryCode, the first comprehensive benchmark designed to span multiple industrial domains and programming languages. IndustryCode comprises 579 sub-problems derived from 125 primary industrial challenges, accompanied by rigorous problem descriptions and test cases. It covers a wide range of fields, including finance, automation, aerospace, and remote sensing-and incorporates diverse programming languages such as MATLAB, Python, C++, and Stata. In our evaluation, the top-performing model, Claude 4.5 Opus, achieved an overall accuracy of 68.1% on sub-problems and 42.5% main problems. The benchmark dataset and automated evaluation code will be made publicly available upon acceptance.

Related papers

• Unification of Closed-Open Industrial Detection Scenarios: New Large-Scale Benchmarks,Challenges and Baselines [50.471211036005286]
  We introduce a Large-Scale Multi-Modal Industrial Open-Closed benchmark (MMIOC-1M)<n> MMIOC-1M is the first unified largest benchmark supporting both open-vocabulary and closed-set industrial detection.<n>We propose a Refined Text-Visual Prompt Network (RTVPNet) that incorporates three key innovations.
  arXiv  Detail & Related papers  (2026-06-06T03:06:10Z)
• InCoder-32B-Thinking: Industrial Code World Model for Thinking [110.60836637653127]
  InCoder-32B-Thinking is trained on the data from the Error-driven Chain-of-Thought (ECoT) synthesis framework.<n>ECoT generates reasoning chains by synthesizing the thinking content from multi-turn dialogue with environmental error feedback.<n>InCoder-32B-Thinking achieves top-tier open-source results across all domains.
  arXiv  Detail & Related papers  (2026-04-03T16:06:25Z)
• InCoder-32B: Code Foundation Model for Industrial Scenarios [95.28947955949296]
  We introduce InCoder-32B (Industrial-Coder-32B) to unify code intelligence across chip design, GPU kernel optimization, embedded systems, compiler optimization, and 3D modeling.<n>InCoder-32B achieves highly competitive performance on general tasks while establishing strong open-source baselines across industrial domains.
  arXiv  Detail & Related papers  (2026-03-17T17:01:35Z)
• Aligning Academia with Industry: An Empirical Study of Industrial Needs and Academic Capabilities in AI-Driven Software Engineering [45.09204791294318]
  The rapid advancement of large language models (LLMs) is fundamentally reshaping software engineering (SE)<n>While top-tier SE venues continue to show sustained or emerging focus on areas like automated testing and program repair, the alignment of these academic advances with real industrial needs remains unclear.<n>This study aims to refocus academic attention on these important yet under-explored problems and to guide future SE research toward greater industrial impact.
  arXiv  Detail & Related papers  (2025-12-17T07:29:18Z)
• SWE-Compass: Towards Unified Evaluation of Agentic Coding Abilities for Large Language Models [59.90381306452982]
  evaluating large language models (LLMs) for software engineering has been limited by narrow task coverage, language bias, and insufficient alignment with real-world developer.<n>We introduce SWE-1, a comprehensive benchmark that unifies heterogeneous code-related evaluations into a structured and production-aligned framework.<n>SWE- spans 8 task types, 8 programming scenarios, and 10 programming languages, with 2000 high-quality instances curated from authentic GitHub pull requests.
  arXiv  Detail & Related papers  (2025-11-07T18:01:32Z)
• MERA Code: A Unified Framework for Evaluating Code Generation Across Tasks [56.34018316319873]
  We propose MERA Code, a benchmark for evaluating code for the latest code generation LLMs in Russian.<n>This benchmark includes 11 evaluation tasks that span 8 programming languages.<n>We evaluate open LLMs and frontier API models, analyzing their limitations in terms of practical coding tasks in non-English languages.
  arXiv  Detail & Related papers  (2025-07-16T14:31:33Z)
• Software Development Life Cycle Perspective: A Survey of Benchmarks for Code Large Language Models and Agents [23.476042888072293]
  Code large language models (CodeLLMs) and agents have shown great promise in tackling complex software engineering tasks.<n>This paper provides a comprehensive review of existing benchmarks for CodeLLMs and agents, studying and analyzing 181 benchmarks from 461 relevant papers.
  arXiv  Detail & Related papers  (2025-05-08T14:27:45Z)
• MME-Industry: A Cross-Industry Multimodal Evaluation Benchmark [20.642661835794975]
  We introduce MME-Industry, a novel benchmark designed specifically for evaluating MLLMs in industrial settings.<n>The benchmark encompasses 21 distinct domain, comprising 1050 question-answer pairs with 50 questions per domain.<n>We provide both Chinese and English versions of the benchmark, enabling comparative analysis of MLLMs' capabilities across these languages.
  arXiv  Detail & Related papers  (2025-01-28T03:56:17Z)
• MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detection [66.05200339481115]
  We present MMAD, the first-ever full-spectrum MLLMs benchmark in industrial anomaly detection.<n>We defined seven key subtasks of MLLMs in industrial inspection and designed a novel pipeline to generate the MMAD dataset.<n>With MMAD, we have conducted a comprehensive, quantitative evaluation of various state-of-the-art MLLMs.
  arXiv  Detail & Related papers  (2024-10-12T09:16:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.