WatChat: Explaining perplexing programs by debugging mental models
- URL: http://arxiv.org/abs/2403.05334v1
- Date: Fri, 8 Mar 2024 14:10:25 GMT
- Title: WatChat: Explaining perplexing programs by debugging mental models
- Authors: Kartik Chandra, Tzu-Mao Li, Rachit Nigam, Joshua Tenenbaum, Jonathan
Ragan-Kelley
- Abstract summary: We use program synthesis techniques to automatically infer potential misconceptions that might cause the user to be surprised by the program's behavior.
By analyzing these misconceptions, we provide succinct, useful explanations of the program's behavior.
Our methods can even be inverted to synthesize pedagogical example programs for diagnosing and correcting misconceptions in students.
- Score: 8.60151271579051
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Often, a good explanation for a program's unexpected behavior is a bug in the
programmer's code. But sometimes, an even better explanation is a bug in the
programmer's mental model of the language they are using. Instead of merely
debugging our current code ("giving the programmer a fish"), what if our tools
could directly debug our mental models ("teaching the programmer to fish")? In
this paper, we apply ideas from computational cognitive science to do exactly
that. Given a perplexing program, we use program synthesis techniques to
automatically infer potential misconceptions that might cause the user to be
surprised by the program's behavior. By analyzing these misconceptions, we
provide succinct, useful explanations of the program's behavior. Our methods
can even be inverted to synthesize pedagogical example programs for diagnosing
and correcting misconceptions in students.
Related papers
- A Proposal for a Debugging Learning Support Environment for Undergraduate Students Majoring in Computer Science [0.0]
Students do not know how to use a debugger or have never used one.
We implemented a function in Scratch that allows for self-learning of correct breakpoint placement.
arXiv Detail & Related papers (2024-07-25T03:34:19Z) - Can humans teach machines to code? [24.32052793811087]
A key underlying assumption is that humans can provide examples of sufficient quality to teach a concept to a machine.
We ask humans to generate examples for six programming tasks, such as finding the maximum element of a list.
We compare the performance of a program synthesis system trained on (i) human-provided examples, (ii) randomly sampled examples, and (iii) expert-provided examples.
arXiv Detail & Related papers (2024-04-30T09:42:40Z) - NExT: Teaching Large Language Models to Reason about Code Execution [50.93581376646064]
Large language models (LLMs) of code are typically trained on the surface textual form of programs.
We propose NExT, a method to teach LLMs to inspect the execution traces of programs and reason about their run-time behavior.
arXiv Detail & Related papers (2024-04-23T01:46:32Z) - What is a "bug"? On subjectivity, epistemic power, and implications for
software research [8.116831482130555]
"Bug" has been a colloquialism for an engineering "defect" at least since the 1870s.
Most modern software-oriented definitions speak to a disconnect between what a developer intended and what a program actually does.
"Finding bugs is easy" begins by saying "bug patterns are code that are often errors"
arXiv Detail & Related papers (2024-02-13T01:52:42Z) - On Feasibility of Declarative Diagnosis [0.0]
We argue that useful ways of declarative diagnosis of logic programs exist, and should be usable in actual programming.
This paper discusses their possibly main weaknesses and shows how to overcome them.
arXiv Detail & Related papers (2023-08-30T08:56:19Z) - Giving Feedback on Interactive Student Programs with Meta-Exploration [74.5597783609281]
Developing interactive software, such as websites or games, is a particularly engaging way to learn computer science.
Standard approaches require instructors to manually grade student-implemented interactive programs.
Online platforms that serve millions, like Code.org, are unable to provide any feedback on assignments for implementing interactive programs.
arXiv Detail & Related papers (2022-11-16T10:00:23Z) - Using Large Language Models to Enhance Programming Error Messages [5.903720638984496]
Large language models can be used to create useful enhancements to programming error messages.
We discuss the benefits and downsides of large language models and highlight future streams of research for enhancing programming error messages.
arXiv Detail & Related papers (2022-10-20T23:17:26Z) - Fault-Aware Neural Code Rankers [64.41888054066861]
We propose fault-aware neural code rankers that can predict the correctness of a sampled program without executing it.
Our fault-aware rankers can significantly increase the pass@1 accuracy of various code generation models.
arXiv Detail & Related papers (2022-06-04T22:01:05Z) - Non-Programmers Can Label Programs Indirectly via Active Examples: A
Case Study with Text-to-SQL [61.950839050383514]
APEL is a framework in which non-programmers select among candidate programs generated by a seed semantic (e.g., Codex)
For each utterance, APEL actively searches for a simple input on which the candidate programs tend to produce different outputs.
It asks the non-programmers only to choose the appropriate output, thus allowing us to infer which program is correct.
arXiv Detail & Related papers (2022-05-25T00:35:12Z) - Searching for More Efficient Dynamic Programs [61.79535031840558]
We describe a set of program transformations, a simple metric for assessing the efficiency of a transformed program, and a search procedure to improve this metric.
We show that in practice, automated search can find substantial improvements to the initial program.
arXiv Detail & Related papers (2021-09-14T20:52:55Z) - Enforcing Consistency in Weakly Supervised Semantic Parsing [68.2211621631765]
We explore the use of consistency between the output programs for related inputs to reduce the impact of spurious programs.
We find that a more consistent formalism leads to improved model performance even without consistency-based training.
arXiv Detail & Related papers (2021-07-13T03:48:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.