SUBMISSION: 1227 TITLE: How Useful are Reviews for Recommendation? A Critical Review and Potential Improvements ----------------------- REVIEW 1 --------------------- SUBMISSION: 1227 TITLE: How Useful are Reviews for Recommendation? A Critical Review and Potential Improvements AUTHORS: Noveen Sachdeva and Julian McAuley ----------- Detailed comments ----------- This paper conducted a comprehensive comparison of recommendation models that exploits user review data. The findings are interesting, not only does it show whether and when user review data would help, but also provide insights into the reproducibility of recent models from the literature. The paper is very well written and easy to follow. An additional plus is the use of public datasets, and the implementations are made public. I think the paper will provoke discussions and have impact on future research in this area. ----------- Recommendation ----------- SCORE: 2 (accept) ----------- Relevance ----------- SCORE: 4 (Paper is on a current topic of interest to a large fraction of the audience) ----------- Originality of the work ----------- SCORE: 3 (Somewhat conventional: A number of people could have come up with this after some thought) ----------------------- REVIEW 2 --------------------- SUBMISSION: 1227 TITLE: How Useful are Reviews for Recommendation? A Critical Review and Potential Improvements AUTHORS: Noveen Sachdeva and Julian McAuley ----------- Detailed comments ----------- The authors argue that the benefit of using reviews for recommendation is overstated, and in particular, the substantial reported gains are only possible under a narrow set of conditions. Pros - the topic is relevant Cons - It is a common sense that reviews are not always useful. It usually varies from domains to domains, data to data. It may not be able to improve the recommendations always, but it is still beneficial from the perspective of other aspects, such as explanation of recommenations, fairness and transparency, and so forth - The authors in this paper focused much more on the rating prediction task which was examined by the MSE metric. However, the effect may be different in the top-N recommendation task. The authors present limited results based on HR@1. However, more results are necessary to deliver a solid conclusion. ----------- Recommendation ----------- SCORE: 0 (borderline) ----------- Relevance ----------- SCORE: 3 (Paper is on a current topic of interest to small fraction of the audience) ----------- Originality of the work ----------- SCORE: 2 (Rather straightforward: Obvious or a minor improvement on familiar techniques) ----------------------- REVIEW 3 --------------------- SUBMISSION: 1227 TITLE: How Useful are Reviews for Recommendation? A Critical Review and Potential Improvements AUTHORS: Noveen Sachdeva and Julian McAuley ----------- Detailed comments ----------- This paper presents a comparison of different recommendation algorithms which are based on exploiting reviews. The paper is written in a very well-structured manner and the comparison is performed in a througouh manner. Authors provide a good discussion of their empirical findings. Majors: * Inappropriate way of providing references (generally no page numbers, no volume/number for journals etc.) Minors: * Spelling/grammar errors (e.g. publically should read publicly) * I would recommend to give the dates of last access for all URLs used in footnotes. ----------- Recommendation ----------- SCORE: 2 (accept) ----------- Relevance ----------- SCORE: 3 (Paper is on a current topic of interest to small fraction of the audience) ----------- Originality of the work ----------- SCORE: 3 (Somewhat conventional: A number of people could have come up with this after some thought)