PizzaCommonSense: Learning to Model Commonsense Reasoning about
Intermediate Steps in Cooking Recipes
- URL: http://arxiv.org/abs/2401.06930v1
- Date: Fri, 12 Jan 2024 23:33:01 GMT
- Title: PizzaCommonSense: Learning to Model Commonsense Reasoning about
Intermediate Steps in Cooking Recipes
- Authors: Aissatou Diallo, Antonis Bikakis, Luke Dickens, Anthony Hunter, Rob
Miller
- Abstract summary: We present a new corpus of cooking recipes enriched with descriptions of intermediate steps of the recipes that explicate the input and output for each step.
This work presents a challenging task and insight into commonsense reasoning and procedural text generation.
- Score: 8.410402833223364
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Decoding the core of procedural texts, exemplified by cooking recipes, is
crucial for intelligent reasoning and instruction automation. Procedural texts
can be comprehensively defined as a sequential chain of steps to accomplish a
task employing resources. From a cooking perspective, these instructions can be
interpreted as a series of modifications to a food preparation, which initially
comprises a set of ingredients. These changes involve transformations of
comestible resources. For a model to effectively reason about cooking recipes,
it must accurately discern and understand the inputs and outputs of
intermediate steps within the recipe. Aiming to address this, we present a new
corpus of cooking recipes enriched with descriptions of intermediate steps of
the recipes that explicate the input and output for each step. We discuss the
data collection process, investigate and provide baseline models based on T5
and GPT-3.5. This work presents a challenging task and insight into commonsense
reasoning and procedural text generation.
Related papers
- Large Language Models as Sous Chefs: Revising Recipes with GPT-3 [56.7155146252028]
We focus on recipes as an example of complex, diverse, and widely used instructions.
We develop a prompt grounded in the original recipe and ingredients list that breaks recipes down into simpler steps.
We also contribute an Amazon Mechanical Turk task that is carefully designed to reduce fatigue while collecting human judgment of the quality of recipe revisions.
arXiv Detail & Related papers (2023-06-24T14:42:43Z) - Counterfactual Recipe Generation: Exploring Compositional Generalization
in a Realistic Scenario [60.20197771545983]
We design the counterfactual recipe generation task, which asks models to modify a base recipe according to the change of an ingredient.
We collect a large-scale recipe dataset in Chinese for models to learn culinary knowledge.
Results show that existing models have difficulties in modifying the ingredients while preserving the original text style, and often miss actions that need to be adjusted.
arXiv Detail & Related papers (2022-10-20T17:21:46Z) - A Rich Recipe Representation as Plan to Support Expressive Multi Modal
Queries on Recipe Content and Preparation Process [24.94173789568803]
We discuss the construction of a machine-understandable rich recipe representation (R3)
R3 is infused with additional knowledge such as information about allergens and images of ingredients.
We also present TREAT, a tool for recipe retrieval which uses R3 to perform multi-modal reasoning on the recipe's content.
arXiv Detail & Related papers (2022-03-31T15:29:38Z) - Revamping Cross-Modal Recipe Retrieval with Hierarchical Transformers
and Self-supervised Learning [17.42688184238741]
Cross-modal recipe retrieval has recently gained substantial attention due to the importance of food in people's lives.
We propose a simplified end-to-end model based on well established and high performing encoders for text and images.
Our proposed method achieves state-of-the-art performance in the cross-modal recipe retrieval task on the Recipe1M dataset.
arXiv Detail & Related papers (2021-03-24T10:17:09Z) - Structure-Aware Generation Network for Recipe Generation from Images [142.047662926209]
We investigate an open research task of generating cooking instructions based on only food images and ingredients.
Target recipes are long-length paragraphs and do not have annotations on structure information.
We propose a novel framework of Structure-aware Generation Network (SGN) to tackle the food recipe generation task.
arXiv Detail & Related papers (2020-09-02T10:54:25Z) - Multi-modal Cooking Workflow Construction for Food Recipes [147.4435186953995]
We build MM-ReS, the first large-scale dataset for cooking workflow construction.
We propose a neural encoder-decoder model that utilizes both visual and textual information to construct the cooking workflow.
arXiv Detail & Related papers (2020-08-20T18:31:25Z) - Decomposing Generation Networks with Structure Prediction for Recipe
Generation [142.047662926209]
We propose a novel framework: Decomposing Generation Networks (DGN) with structure prediction.
Specifically, we split each cooking instruction into several phases, and assign different sub-generators to each phase.
Our approach includes two novel ideas: (i) learning the recipe structures with the global structure prediction component and (ii) producing recipe phases in the sub-generator output component based on the predicted structure.
arXiv Detail & Related papers (2020-07-27T08:47:50Z) - A Recipe for Creating Multimodal Aligned Datasets for Sequential Tasks [48.39191088844315]
In the cooking domain, the web offers many partially-overlapping text and video recipes that describe how to make the same dish.
We use an unsupervised alignment algorithm that learns pairwise alignments between instructions of different recipes for the same dish.
We then use a graph algorithm to derive a joint alignment between multiple text and multiple video recipes for the same dish.
arXiv Detail & Related papers (2020-05-19T17:27:00Z) - Classification of Cuisines from Sequentially Structured Recipes [8.696042114987966]
classification of cuisines based on their culinary features is an outstanding problem.
We have implemented a range of classification techniques by accounting for this information on the RecipeDB dataset.
The state-of-the-art RoBERTa model presented the highest accuracy of 73.30% among a range of classification models.
arXiv Detail & Related papers (2020-04-26T05:40:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.