The Proof is in the Almond Cookies

• URL: http://arxiv.org/abs/2501.01827v1
• Date: Fri, 03 Jan 2025 14:25:35 GMT
• Title: The Proof is in the Almond Cookies
• Authors: Remi van Trijp, Katrien Beuls, Paul Van Eecke,
• Abstract summary: This paper presents a case study on how to process cooking recipes (and more generally, how-to instructions) in a way that makes it possible for a robot or artificial cooking assistant to support human chefs in the kitchen. We propose a novel approach to computational recipe understanding that mimics the human sense-making process, which is narrative-based.
• Score: 7.534061469399505
• License:
• Abstract: This paper presents a case study on how to process cooking recipes (and more generally, how-to instructions) in a way that makes it possible for a robot or artificial cooking assistant to support human chefs in the kitchen. Such AI assistants would be of great benefit to society, as they can help to sustain the autonomy of aging adults or people with a physical impairment, or they may reduce the stress in a professional kitchen. We propose a novel approach to computational recipe understanding that mimics the human sense-making process, which is narrative-based. Using an English recipe for almond crescent cookies as illustration, we show how recipes can be modelled as rich narrative structures by integrating various knowledge sources such as language processing, ontologies, and mental simulation. We show how such narrative structures can be used for (a) dealing with the challenges of recipe language, such as zero anaphora, (b) optimizing a robot's planning process, (c) measuring how well an AI system understands its current tasks, and (d) allowing recipe annotations to become language-independent.

Related papers

• CookingDiffusion: Cooking Procedural Image Generation with Stable Diffusion [58.92430755180394]
  We present textbfCookingDiffusion, a novel approach to generate photo-realistic images of cooking steps. These prompts encompass text prompts, image prompts, and multi-modal prompts, ensuring the consistent generation of cooking procedural images. Our experimental results demonstrate that our model excels at generating high-quality cooking procedural images.
  arXiv  Detail & Related papers  (2025-01-15T06:58:53Z)
• PizzaCommonSense: Learning to Model Commonsense Reasoning about Intermediate Steps in Cooking Recipes [7.839338724237275]
  A model to effectively reason about cooking recipes must accurately discern and understand the inputs and outputs of intermediate steps within the recipe. We present a new corpus of cooking recipes enriched with descriptions of intermediate steps that describe the input and output for each step.
  arXiv  Detail & Related papers  (2024-01-12T23:33:01Z)
• Towards End-User Development for IoT: A Case Study on Semantic Parsing of Cooking Recipes for Programming Kitchen Devices [4.863892359772122]
  We provide a unique corpus which aims to support the transformation of cooking recipe instructions to machine-understandable commands for IoT devices in the kitchen. Based on this corpus, we developed machine learning-based sequence labelling methods, namely conditional random fields (CRF) and a neural network model. Our results show that while it is feasible to train semantics based on our annotations, most natural-language instructions are incomplete, and thus transforming them into formal meaning representation is not straightforward.
  arXiv  Detail & Related papers  (2023-09-25T14:21:24Z)
• Large Language Models as Sous Chefs: Revising Recipes with GPT-3 [56.7155146252028]
  We focus on recipes as an example of complex, diverse, and widely used instructions. We develop a prompt grounded in the original recipe and ingredients list that breaks recipes down into simpler steps. We also contribute an Amazon Mechanical Turk task that is carefully designed to reduce fatigue while collecting human judgment of the quality of recipe revisions.
  arXiv  Detail & Related papers  (2023-06-24T14:42:43Z)
• Task Tree Retrieval for Robotic Cooking [0.0]
  In this essay, the idea of FOON, a structural knowledge representation built on insights from human manipulations, is introduced. To reduce the failure rate and ensure that the task is effectively completed, three different algorithms have been implemented. This knowledge representation was created using videos of open-sourced recipes.
  arXiv  Detail & Related papers  (2022-11-03T12:18:31Z)
• Counterfactual Recipe Generation: Exploring Compositional Generalization in a Realistic Scenario [60.20197771545983]
  We design the counterfactual recipe generation task, which asks models to modify a base recipe according to the change of an ingredient. We collect a large-scale recipe dataset in Chinese for models to learn culinary knowledge. Results show that existing models have difficulties in modifying the ingredients while preserving the original text style, and often miss actions that need to be adjusted.
  arXiv  Detail & Related papers  (2022-10-20T17:21:46Z)
• Assistive Recipe Editing through Critiquing [34.1050269670062]
  RecipeCrit is a hierarchical denoising auto-encoder that edits recipes given ingredient-level critiques. Our work's main innovation is our unsupervised critiquing module that allows users to edit recipes by interacting with the predicted ingredients.
  arXiv  Detail & Related papers  (2022-05-05T05:52:27Z)
• A Rich Recipe Representation as Plan to Support Expressive Multi Modal Queries on Recipe Content and Preparation Process [24.94173789568803]
  We discuss the construction of a machine-understandable rich recipe representation (R3) R3 is infused with additional knowledge such as information about allergens and images of ingredients. We also present TREAT, a tool for recipe retrieval which uses R3 to perform multi-modal reasoning on the recipe's content.
  arXiv  Detail & Related papers  (2022-03-31T15:29:38Z)
• Structure-Aware Generation Network for Recipe Generation from Images [142.047662926209]
  We investigate an open research task of generating cooking instructions based on only food images and ingredients. Target recipes are long-length paragraphs and do not have annotations on structure information. We propose a novel framework of Structure-aware Generation Network (SGN) to tackle the food recipe generation task.
  arXiv  Detail & Related papers  (2020-09-02T10:54:25Z)
• Multi-modal Cooking Workflow Construction for Food Recipes [147.4435186953995]
  We build MM-ReS, the first large-scale dataset for cooking workflow construction. We propose a neural encoder-decoder model that utilizes both visual and textual information to construct the cooking workflow.
  arXiv  Detail & Related papers  (2020-08-20T18:31:25Z)
• A Benchmark for Structured Procedural Knowledge Extraction from Cooking Videos [126.66212285239624]
  We propose a benchmark of structured procedural knowledge extracted from cooking videos. Our manually annotated open-vocabulary resource includes 356 instructional cooking videos and 15,523 video clip/sentence-level annotations.
  arXiv  Detail & Related papers  (2020-05-02T05:15:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.