----------------------- REVIEW 1 --------------------- PAPER: 114 TITLE: Modeling Consumer Preferences and Price-Sensitivities from Large-Scale Grocery Shopping Transaction Logs AUTHORS: Mengting Wan, Di Wang, Matt Goldman, Matt Taddy, Justin Rao, Jie Liu, Dimitrios Lymberopoulos and Julian McAuley Originality: 3 Impact: 3 Reproducibility: 3 Overall evaluation: 3 ----------- Strong Points ----------- 1. Well written paper 2. Interesting way of combining price sensitivity into product recommendations 2. Some very interesting insights like promotions work only if a customer is interested in the corresponding product category ----------- Weak Points ----------- 1. Although the paper claims that the technique is scalable, but experients are over few hundred thousand products. It's not very clear how scalable is the technique for millions of products that retailers/ecommerce companies sell these days. 2. Although interesting work, it's incremental work ----------- Detailed Review ----------- This paper talks about recommending category of products, products and quantity, while taking care of price sensitiviy. This paper proposes a model that takes into account both preferences and price sensitivities. The model is personalized, interpretable and scalable. The paper also comes up with interesting insights from their experiments such as (1) price does not affect categories people are not interested in; (2) price is an important factor for product selection, and promotions should be targeted for users interested in that product category. This paper proposes a nested feature based matrix factorization framework for (1) category purchases using a binary prediction model for each category, (2) a model using multinomial distribution using softmax for product selection, and (3) a model using Poisson distribution for purchase quantity. ----------------------- REVIEW 2 --------------------- PAPER: 114 TITLE: Modeling Consumer Preferences and Price-Sensitivities from Large-Scale Grocery Shopping Transaction Logs AUTHORS: Mengting Wan, Di Wang, Matt Goldman, Matt Taddy, Justin Rao, Jie Liu, Dimitrios Lymberopoulos and Julian McAuley Originality: 4 Impact: 2 Reproducibility: 3 Overall evaluation: -2 ----------- Strong Points ----------- - use of multiple datasets in analysis, one of which is a public dataset. - intersting and relevant topic ----------- Weak Points ----------- - facts around the influence of price on shoppers in the grocery setting are missing to motivate this work. This is confounded by the results that including price in the model does not improve its predictive performance. - details of dataset, and price variability are not included, leading to a lack of clarity on the need for price to be factored in, or the limitations of the datasets used. ----------- Detailed Review ----------- The paper introduction and motivation behind the paper should be more clearly articulate in the introduction. Introducing the framework twice in the intro is not necessary. A discussion on how price influences shoppers would be of more value to readers. The last two "Contributions" are not phrased as such. In the related work the authors point to work, but do not sufficiently explain the contributions of the work or how their outcomes are influencing this research. Section 3 has only one subsection, so the subsection is redundant. Consider combining sections 2 and 3 perhaps. More details on the price variability of the datasets is required to understand the datasets. What types of discounts/specials were observed, what relative decreases in price were achieved. Can the authors provide information on the diversity of the shopping habits of the users. Do some users have low levels of product diversity and others appear to be more price driven? The analysis considers single shopping episodes. Given that shoppers have irregular habits (some shop weekly, others several times per week, is it realistic to predict the next shop, from the last? The authors claim that their categories are "fine grained" and should provide details of the number of categories. Why the random split into test and training, in particular if you are not doing a cross-fold validation? A more realistic scenario would be to split the data on chronological order. This is what a real retailer would have? The authors have the transaction logs for the shoppers, but not a log showing the prices of each product at a given time during the data collection period. There may be many products on special or better specials that those observed through the prices paid. How are the authors compensating for this blind spot in their data and assumptions. The authors discuss the relationship between categories and the impact of price variability. Could this be down to the nature of the category? i.e. certain categories include staple items (milk, bread etc) which are required and users might simply automate these purchased. Items that have a higher cost and are purchased less frequently (detergent etc) may prompt more consideration. The overall frequency of purchase could be used to determine regular from irregular items, staple versus discretionary items. ----------------------- REVIEW 3 --------------------- PAPER: 114 TITLE: Modeling Consumer Preferences and Price-Sensitivities from Large-Scale Grocery Shopping Transaction Logs AUTHORS: Mengting Wan, Di Wang, Matt Goldman, Matt Taddy, Justin Rao, Jie Liu, Dimitrios Lymberopoulos and Julian McAuley Originality: 4 Impact: 3 Reproducibility: 3 Overall evaluation: 3 ----------- Strong Points ----------- The paper covers a relevant problem by combining two research areas. It is well written and based on 2 extensive data sets, one publicly available, the other not. The public one is used to ensure reproducibility of the results. ----------- Weak Points ----------- It would been interesting to see a discussion how the developed models translate to non-grocery shopping, e.g. while the authors find (on the grocery data set) that price elasticity for category purchase prediction is negligible, it's not clear to me that this would translate to other product areas. ----------- Detailed Review ----------- The authors model consumer preference in combination with price sensitivity, using two large grocery shopping data set (one public, the other private. The public one is used to ensure reprodusibility of results). While each problem per se has been studied separately before, the contribution of the authors is to combine the two problems. The problem of finding the preferences of a consumer is important in offline and online shopping. As we also see that consumers are sensitive to prices, studying the two problems in combination (preference & price sensitivity) is relevant. The authors start by dividing the shopping decision into 3 stages: category purchase decision, product choice, and product quantity. First logistic probabilities for each stage are derived, then -as it is assumed that each stage’s parameter are independent- models for each stage can be inferred separately using the principle of maximum likelihood estimation. This is actually quite nice, as it allows for more flexible usage of the methodology (one could imagine, depending on the use case, to only use the category-preference modelling) The authors then add the concept of price elasticity to the mix and give elasticity functions for each stage. Given this framework of user behavior for each stage, the authors use a large dataset to evaluate their framework. The paper is overall nicely written, the case study on bacon was interesting to read, and the related work cites relevant contemporary work. I would be interested in seeing how the learnings from grocery shopping data translate to e.g. online shopping in general, or apparel items. ----------------------- REVIEW 4 --------------------- PAPER: 114 TITLE: Modeling Consumer Preferences and Price-Sensitivities from Large-Scale Grocery Shopping Transaction Logs AUTHORS: Mengting Wan, Di Wang, Matt Goldman, Matt Taddy, Justin Rao, Jie Liu, Dimitrios Lymberopoulos and Julian McAuley Originality: 4 Impact: 3 Reproducibility: 3 Overall evaluation: 3 ----------- Strong Points ----------- - The research addresses a topic rarely explored in recommender systems, which is considering prices within the preference model. Although the task evaluated is not exactly recommendation, it can be easily extended based on the results shown. - Document very well written and structured - Technically, the paper is very good, provides a novel approach with all the details necessary to replicate it. - The evaluation in three stages is very interesting and unsual for recommender system community. Using a public dataset (Dunnhumby) along a private one is also good for replicability. Good sensitivity analysis of different methods, metrics and parameters, overall and in the context of a specific category (bacon) - In addition to the model, the article contributes interesting insights about the relation of product categories, user preferences and price. ----------- Weak Points ----------- - Topic slightly out of scope: modelling user behaviour on log transactions at stores and considering price elasticity is not directly benefiting the research community on the World Wide Web. - The evaluation protocol considers a random split of shopping trips in 50/50, but it would have been more realistic doing a chronlogical split. Moreover, further analysis on how the model was affected by the amount of user history would have been interesting. - The authors do not justify certain parameters used, such as why using the dimension of the latent user and item vectors to 5. Maybe that could have had an influence on the prediction, since it seems a rather small value. ----------- Detailed Review ----------- The main contribution of this research is introducing a models that allows combining user preferences from shopping transaction logs considering price elasticity. The model is described in detail, is used to analyze two dataset (one private, another public) and it provides interesting conclusions regarding user intentions, product categories and the influence of prices on customer shopping decisions. Although the model is based on the the recently published GLMIX, the problem has been rarely studied in the context of recommender systems. This makes it novel and it also has large potential to impact in the intersection of marketing and recommender systems. Detailed comments on some points: - The research addresses a topic very rarely explored in recommender systems, which is considering prices within the preference model. The auhors cite some related work but along describing it well, they clarify differences with these previous approaches - The evaluation in three stages is very complete. Using a public dataset (Dunnhumby) along a private one is also good for replicability. Good sensitivity analysis of different methods, metrics and parameters, overall and in the context of a specific category (bacon). Particularly the example on Bacon which allows to dig dipper into the relation between price ellasticity and other variables. - In addition to the model, the article contributes interesting insights about the relation of product categories, user preferences and price. These insights relating preference vs. representative features, relation between personalized product and specific price sensitivity, and preference vs. price sensitivity can give light to additional research questions for further investigation. CONS: - Topic slightly out of scope: modelling user behaviour on log transactions at stores and considering price elasticity is not directly benefiting the research community on the World Wide Web. - The evaluation protocol considers a random split of shopping trips in 50/50, but it would have been more realistic doing a chronlogical split. Further analysis on how the model was affected by the amount of user history would have been interesting. An approach used by users studying lists of recommendations in shopping transactions ("On the Value of Reminders within E-Commerce Recommendations", UMAP 2016) would have been interesting to be used in order to analyze other important factors beyond the relation of price. - The authors do not justify certain parameters used, such as why using the dimension of the latent user and item vectors to 5. Maybe that could have had an influence on the prediction, since it seems a rather small value. Authors disclose these values but they do not provide details of potential limitations or biases in the evaluation to set the number of latent dimensions only to 5, when many articles calculate them in the order or 40-100 latent dimensions. ------------------------- METAREVIEW ------------------------ PAPER: 114 TITLE: Modeling Consumer Preferences and Price-Sensitivities from Large-Scale Grocery Shopping Transaction Logs All reviewers liked the paper and that it addresses a topic which is underexplored in recommender systems, which is considering prices within the preference model. Besides the novelty of the proposed task/solution, they also noted that the paper contains a number of useful insights about the connections between product categories, user preferences and product prices. However, the following issues need to be addressed in the next version of the paper. - clarify about the scalability the proposed framework - add more details on the datasets - clarify about the lack of a log/analysis showing the prices of each product at a given time during the data collection - discuss how our lessons learned based on grocery shopping data translate to other domains - provide results for a chronological split and/or explain why the conclusions made with a random split are valid