============================================================================ EMNLP-IJCNLP 2019 Reviews for Submission #1255 ============================================================================ Title: Generating Personalized Recipes from Historical User Preferences Authors: Bodhisattwa Prasad Majumder, Shuyang Li, Jianmo Ni and Julian McAuley ============================================================================ META-REVIEW ============================================================================ Comments: This paper addresses an interesting and relatively novel problem, which all reviewers appreciated. Some reviewers had issues with the details of the evaluation -- the authors have presented additional evaluation results in the response which, if included in the final paper, would definitely make this a solid contribution to the conference. ============================================================================ REVIEWER #1 ============================================================================ What is this paper about, what contributions does it make, and what are the main strengths and weaknesses? --------------------------------------------------------------------------- The paper discusses a system the applies personalization to recipe generation. I found this application to be interesting and potentially useful, but the result metrics are somewhat mixed. --------------------------------------------------------------------------- Reasons to accept --------------------------------------------------------------------------- The application of personalization based on model of existing recipes and on the user's recent recipes is in interesting extension of previous work. --------------------------------------------------------------------------- Reasons to reject --------------------------------------------------------------------------- While the generated texts are interesting, the result metrics are somewhat disappointing. It would be good to give a fuller explanation of the lower BLEU-4/ROUGE-L scores and the disparity of the results of the three prior-* approaches (Table 2). --------------------------------------------------------------------------- --------------------------------------------------------------------------- Reviewer's Scores --------------------------------------------------------------------------- Overall Recommendation: 3.5 Questions for the Author(s) --------------------------------------------------------------------------- How is your approach more effective than a good search mechanism over a large recipe database? Could you also learn calorie level preferences for each user? Can you give a fuller explanation of the mixed results in table 2? --------------------------------------------------------------------------- Typos, Grammar, Style, and Presentation Improvements --------------------------------------------------------------------------- 013 - "existing approaches" to what? Be clear here. 020 (and elsewhere, e.g., 115, 127, ...) - I found the use of "attend on/over X" throughout the paper confusing. Are you saying that you use attention mechanisms to focus on X? 176 - From where do you get your testing recipes? 346 - I found the rows in Table 3 to be confusing. I suggest adding more detail on what each row is. There should be a conclusion section. --------------------------------------------------------------------------- ============================================================================ REVIEWER #2 ============================================================================ What is this paper about, what contributions does it make, and what are the main strengths and weaknesses? --------------------------------------------------------------------------- Contributions: — New task of user personalized recipe generation — New models for incorporating user preference in three different ways: attending to the names of previous recipes shared by that user as additional input, attending to techniques used by that user in previous recipes as an additional input Strengths: — Motivation is strong. User preferences are an important element to recipe generation as the same goal could be implemented differently based on user restrictions (i.e., low-calorie, gluten-free, vegetarian, etc.) — The authors provide a new, large-scale dataset that could be used for recipe generation with user preferences — The user matching evaluation is convincing and seems to show that there are personalization correlations between users and the recipes they generate, and the model is learning to pick up on this information Weaknesses: — It’s not clear what the difference between the Prior Recipe and Prior Name models are — It’s not clear how integrating the calorie level as an input is useful — The human evaluation should have been run with a “No Preference” option, or else workers are forced to choose one or there other when both recipes could be equally bad. --------------------------------------------------------------------------- Reasons to accept --------------------------------------------------------------------------- Work on user preferences is popular in industry applications, and this work is an interesting take on how to augment the task of recipe generation with that theme --------------------------------------------------------------------------- Reasons to reject --------------------------------------------------------------------------- The paper could have its main contributions written more clearly --------------------------------------------------------------------------- --------------------------------------------------------------------------- Reviewer's Scores --------------------------------------------------------------------------- Overall Recommendation: 4 Missing References --------------------------------------------------------------------------- https://arxiv.org/abs/1805.03766 --------------------------------------------------------------------------- Typos, Grammar, Style, and Presentation Improvements --------------------------------------------------------------------------- The task is not necessarily clear in some parts. I understand this is a short paper, but it would be good if the authors could outline clearly what a single data point might look like. I’m not sure whether a particular recipe might have multiple users making it? Or whether each recipe is associated with one user? Having an “Task Outline” paragraph would clarify these details. --------------------------------------------------------------------------- ============================================================================ REVIEWER #3 ============================================================================ What is this paper about, what contributions does it make, and what are the main strengths and weaknesses? --------------------------------------------------------------------------- The paper proposes an approach for personalized recipe generation. From a recipe name and an incomplete list of ingredients, the system builds a complete recipe aligned with the user's historical preferences. --------------------------------------------------------------------------- Reasons to accept --------------------------------------------------------------------------- The problem addressed by the paper is interesting and appropriate for the conference. --------------------------------------------------------------------------- Reasons to reject --------------------------------------------------------------------------- However, the approach is not clearly presented. For example, it is not clear that the generated recipes will be useful for the user, as they may not be coherent. In addition, the evaluation is not very appropriate. Metrics like BLEU and ROUGE can give an idea about the structural information of the final recipes, but not about other things that are important in this case: order of events, coherence of the steps, usage of all the ingredients,... In addition, the evaluation is not explained in detail, and the reader must go to the suplementary material to understand some aspects of it. --------------------------------------------------------------------------- --------------------------------------------------------------------------- Reviewer's Scores --------------------------------------------------------------------------- Overall Recommendation: 3