Congratulations! Your paper, “Representing and Recommending Shopping Baskets with Complementarity, Compatibility, and Loyalty” (submission number 1551), has been accepted for inclusion in the CIKM 2018 conference. We received 862 full paper submissions, out of which 147 were accepted (17%). The reviews and metareview of your paper are at the bottom of this email. A separate email will be sent with instructions for preparation of the camera-ready version of the paper. Please take full account of the reviewers' comments when preparing the camera-ready version. CIKM requires that one of the authors of your paper attend the conference and present your work. Please also consider attending tutorials and workshops in the conference (http://www.cikm2018.units.it/). Best regards, James Allan, Norman Paton, Divesh Srivastava CIKM 2018 Long Paper Track Co-Chairs ----------------------- REVIEW 1 --------------------- PAPER: 1551 TITLE: Representing and Recommending Shopping Baskets with Complementarity, Compatibility, and Loyalty AUTHORS: Mengting Wan, Di Wang, Jie Liu, Paul Bennett and Julian McAuley Relevance to CIKM: 5 (excellent) Originality of the Work: 4 (good) Technical Soundness: 4 (good) Quality of Presentation: 4 (good) Impact of Ideas or Results: 3 (fair) Adequacy of Citations: 3 (fair) Reproducibility of Methods: 4 (good) Overall Evaluation: -2 (I am not championing but if there is a champion then I am fine accepting) ----------- List 3 or more strong points, labelled S1, S2, ... ----------- S1: good practical scope of work, which to my knowledge has not been extensively studied (grocery shopping) in our community S2: Diverse set of datasets used for evaluation (four in total). S3: Good set of state-of-the-art baselines. S4: Broad set of results consistently show the superiority of the proposed algorithms. ----------- List 3 or more weak points, labelled W1, W2, ... ----------- W1: Lacking definitions. I would not call preferences matching "compatibility" but rather "personalization" or something like that. Loyalty is also not the best choice of terminology in my opinion for repeated purchases. Also, while I agree that Loyalty characterize grocery buying, I disagree "compatibility" is unique to the domain. While in grocery, repeated purchases are common, loyalty to brands is common across many categories (fashion, electronics, etc.) W2: the tie between the unique characteristics of grocery shopping to the proposed triple2vec framework is not convincing. In particular, why is this method especially suitable for grocery while others (item2vec, prod2vec) are suitable for the rest? In a sense, each shopping category (fashion, electronics, home) has its own unique characteristics. W3: Datasets represent specialize and relatively small groceries, each with very different characteristics. Not clear how the results generalize. ----------- Overall Evaluation ----------- A few potential extensions the authors may consider: - Sharpen the definitions of Grocery shopping. I think most emphasis should be put on repeated (periodic) purchases. - Evaluate the approach on other domains and compare with the results for Grocery. - Perform hyper-param tuning, e.g., for embedding size. ----------------------- REVIEW 2 --------------------- PAPER: 1551 TITLE: Representing and Recommending Shopping Baskets with Complementarity, Compatibility, and Loyalty AUTHORS: Mengting Wan, Di Wang, Jie Liu, Paul Bennett and Julian McAuley Relevance to CIKM: 5 (excellent) Originality of the Work: 4 (good) Technical Soundness: 4 (good) Quality of Presentation: 5 (excellent) Impact of Ideas or Results: 5 (excellent) Adequacy of Citations: 4 (good) Reproducibility of Methods: 4 (good) Overall Evaluation: 6 (I fully champion and expect this to be in the top 10% of papers in this track) ----------- List 3 or more strong points, labelled S1, S2, ... ----------- S1 - Novel product recommendation algorithms that capture inherent patterns of grocery stores transaction data; S2 - The proposed algorithms are sound and built on top of solid theory and algorithms; S3 - The experiments are thorough and the results are good. ----------- List 3 or more weak points, labelled W1, W2, ... ----------- W1 - Some parts of the approaches lack further details (see more on this below); W2 - I missed some discussion about the scalability of the proposed approaches. Are they suitable for real-time recommendations? For example, is it feasible to provide recommendations while a customer is shopping in a grocery store? W3 - The same as W2 but considering sparsity. I missed some discussion on how the proposed approaches behave in highly sparse scenarios. ----------- Overall Evaluation ----------- This paper exploits different kinds of patterns that appear often in grocery store transaction data, namely, , complementarity compatibility and loyalty. The authors propose new algorithms inspired on word2vec for recommending next-basket and within-basket products. The paper is very well written and addresses a recommendation scenario scarcely studied in recommender systems, i.e., most of the recommender systems literature focus on e-commerce related domains. The experiments are thorough and provides plenty of evidence and insights about the quality of the proposed approaches, considering different real-world datasets and strong baselines. The paper could be more self-contained in some parts. For example, in "The associated weights are learned by applying the same pairwise ranking loss (i.e., the BPR loss [26])" the procedure could be more explicit instead of leaving to the reader to wonder how this is done exactly. Discussions/experiments about the complexity and scalability of the approach, as well as its feasibility for real-time recommendations, would be nice to have. The same comment goes for highly sparse scenarios. ----------------------- REVIEW 3 --------------------- PAPER: 1551 TITLE: Representing and Recommending Shopping Baskets with Complementarity, Compatibility, and Loyalty AUTHORS: Mengting Wan, Di Wang, Jie Liu, Paul Bennett and Julian McAuley Relevance to CIKM: 4 (good) Originality of the Work: 3 (fair) Technical Soundness: 3 (fair) Quality of Presentation: 4 (good) Impact of Ideas or Results: 3 (fair) Adequacy of Citations: 4 (good) Reproducibility of Methods: 4 (good) Overall Evaluation: -2 (I am not championing but if there is a champion then I am fine accepting) ----------- List 3 or more strong points, labelled S1, S2, ... ----------- S1: The paper proposes a representation learning model triple2vec to learn the complementarity and compatibility and among items for shopping application. S2: the paper proposed adaLoyal, an algorithm for personalized grocery recommendation. S3: The paper presents experiments results conducted on two public and two proprietary datasets, ----------- List 3 or more weak points, labelled W1, W2, ... ----------- W1: The difference in performance for classification between triple2vec and item2vec is not that much W2: The difference in performance adaLoyal and PBR is not that much ----------- Overall Evaluation ----------- The paper proposes a representation learning method called triple2vec, to learn complementarity and compatibility, and an algorithm adaLoyal for product recommendation by adaptively balancing universal product embeddings and users’ product loyalty over time. The effectiveness of both the representation and the proposed algorithm evaluated using two public and two proprietary grocery datasets. The main problem is that the difference in performance for classification between triple2vec and item2vec is not that much although the paper claims that triple2vec substantially and consistently outperforms all baselines on both department and category classification, the difference is not that important. The same observation for the recommendation experiment: the difference between adaLoyal and BPR is not that much ------------------------- METAREVIEW ------------------------ PAPER: 1551 TITLE: Representing and Recommending Shopping Baskets with Complementarity, Compatibility, and Loyalty The referees indicate several strong points. They also indicate several points for improvement such as: Application in other domains. Differences in performance: are they really different? Complexity, scalability, and feasibility for real-time recommendations. It would be nice to see reflection from them in your work. Thanks for considering CIKM for publication.