Compilation Quotient (CQ): A Metric for the Compilation Hardness of Programming Languages

• URL: http://arxiv.org/abs/2406.04778v1
• Date: Fri, 7 Jun 2024 09:28:05 GMT
• Title: Compilation Quotient (CQ): A Metric for the Compilation Hardness of Programming Languages
• Authors: Vince Szabo, Dominik Winterer, Zhendong Su,
• Abstract summary: A key aspect of a programming language's complexity is how hard it is to compile programs. We introduce the compilation quotient (CQ), a metric to quantify compilation hardness. C has a CQ of 48.11, C++ has 0.60, Java has 0.27 and Haskell has 0.13.
• Score: 5.964023099122235
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Today's programmers can choose from an exceptional range of programming languages, each with its own traits, purpose, and complexity. A key aspect of a language's complexity is how hard it is to compile programs in the language. While most programmers have an intuition about compilation hardness for different programming languages, no metric exists to quantify it. We introduce the compilation quotient (CQ), a metric to quantify the compilation hardness of compiled programming languages. The key idea is to measure the compilation success rates of programs sampled from context-free grammars. To this end, we fairly sample over 12 million programs in total. CQ ranges between 0 and 100, where 0 indicates that no programs compile, and 100 means that all programs compile. Our findings on 12 popular compiled programming languages show high variation in CQ. C has a CQ of 48.11, C++ has 0.60, Java has 0.27 and Haskell has 0.13. Strikingly, Rust's CQ is nearly 0, and for C, even a large fraction of very sizable programs compile. We believe CQ can help understand the differences of compiled programming languages better and help language designers.

Related papers

• CRUXEval-X: A Benchmark for Multilingual Code Reasoning, Understanding and Execution [50.7413285637879]
  The CRUXEVAL-X code reasoning benchmark contains 19 programming languages. It comprises at least 600 subjects for each language, along with 19K content-consistent tests in total. Even a model trained solely on Python can achieve at most 34.4% Pass@1 in other languages.
  arXiv  Detail & Related papers  (2024-08-23T11:43:00Z)
• PTE: Axiomatic Semantics based Compiler Testing [7.203331838793759]
  We propose an axiomatic semantics based approach for testing compilers, called PTE. The idea is to incrementally develop a set of axioms'' capturing anecdotes of the language semantics in the form of emph(textbfprecondition, textbftransformation, textbfexpectation''
  arXiv  Detail & Related papers  (2024-01-02T04:50:47Z)
• Creation of a CS1 Course with Modern C++ Principles [0.30458514384586394]
  This poster contains experiences in designing a CS1 course that uses the C++ programming language. Our goals were to prevent many common bad habits among C++ programmers.
  arXiv  Detail & Related papers  (2023-11-13T23:43:38Z)
• Natural Language Embedded Programs for Hybrid Language Symbolic Reasoning [84.12154024070024]
  We propose natural language embedded programs (NLEP) as a unifying framework for addressing math/symbolic reasoning, natural language understanding, and instruction following tasks. Our approach prompts a language model to generate full Python programs that define functions over data structures which contain natural language representations of structured knowledge. A Python interpreter then executes the generated code and prints the output.
  arXiv  Detail & Related papers  (2023-09-19T17:54:21Z)
• Natural Language to Code Translation with Execution [82.52142893010563]
  Execution result--minimum Bayes risk decoding for program selection. We show that it improves the few-shot performance of pretrained code models on natural-language-to-code tasks.
  arXiv  Detail & Related papers  (2022-04-25T06:06:08Z)
• AVATAR: A Parallel Corpus for Java-Python Program Translation [77.86173793901139]
  Program translation refers to migrating source code from one language to another. We present AVATAR, a collection of 9,515 programming problems and their solutions written in two popular languages, Java and Python.
  arXiv  Detail & Related papers  (2021-08-26T05:44:20Z)
• Extending C++ for Heterogeneous Quantum-Classical Computing [56.782064931823015]
  qcor is a language extension to C++ and compiler implementation that enables heterogeneous quantum-classical programming, compilation, and execution in a single-source context. Our work provides a first-of-its-kind C++ compiler enabling high-level quantum kernel (function) expression in a quantum-language manner.
  arXiv  Detail & Related papers  (2020-10-08T12:49:07Z)
• SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection [81.85463892070085]
  The SIGMORPHON 2020 task on morphological reinflection aims to investigate systems' ability to generalize across typologically distinct languages. Systems were developed using data from 45 languages and just 5 language families, fine-tuned with data from an additional 45 languages and 10 language families (13 in total), and evaluated on all 90 languages.
  arXiv  Detail & Related papers  (2020-06-20T13:24:14Z)
• Quantifying the Impact on Software Complexity of Composable Inductive Programming using Zoea [0.0]
  Composable inductive programming as implemented in the Zoea programming language is a simple declarative approach to software development. This paper presents the results of a quantitative comparison of the software complexity of equivalent code implemented in Zoea and also in a conventional programming language.
  arXiv  Detail & Related papers  (2020-05-17T10:44:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.