------------------------- METAREVIEW ------------------------ All reviewers find this paper interesting and solid. However, they also raised concerns about some details in the paper. Please address these comments in the camera ready version. ----------------------- REVIEW 1 --------------------- SUBMISSION: 6764 TITLE: Research: Query-Aware Sequential Recommendation AUTHORS: Zhankui He, Handong Zhao, Zhaowen Wang, Zhe Lin, Ajinkya Kale and Julian Mcauley ----------- Overall evaluation ----------- SCORE: 1 (weak accept) ----- TEXT: The authors propose a query-aware model for sequential recommendation that considers user queries as a contextual feature. Query-SeqRec has a transformer architecture and exploits heterogeneous user sequences (made of user-item interactions and user queries). These sequences are augmented during training using a graph-based approach that stochastically replaces items/queries with semantically similar items/queries. The authors propose to use sampled softmax loss to handle a large number of item embeddings. It would be interesting to compare with: Jiancan Wu, Xiang Wang, Xingyu Gao, Jiawei Chen, Hongcheng Fu, Tianyu Qiu, Xiangnan He: On the Effectiveness of Sampled Softmax Loss for Item Recommendation. CoRR abs/2201.02327 (2022) The authors use two public datasets and an industrial dataset with 10 million items. ----------- Strengths and reasons to accept ----------- - Ablation study for the effectiveness of query information and sequence augmentation. - Exploration of different approaches to incorporate user queries into item interaction sequences (Table 4). - Use of strong sequence model baselines (SASRec and BERT4Rec). - Code will be released upon acceptance. ----------- Weaknesses and limitations ----------- - It is not clear if the the industrial dataset will be released. - The model description (Section 2.3) is a bit condensed. More details to compare with BERT4Rec or Transformers4Rec would be useful. ----------------------- REVIEW 2 --------------------- SUBMISSION: 6764 TITLE: Research: Query-Aware Sequential Recommendation AUTHORS: Zhankui He, Handong Zhao, Zhaowen Wang, Zhe Lin, Ajinkya Kale and Julian Mcauley ----------- Overall evaluation ----------- SCORE: 2 (accept) ----- TEXT: The paper proposes a query-aware sequential recommendation system. Instead of focusing only on history of user actions and information on clicks, content etc., the authors argue user queries also carry important context. They propose a model to incorporate query information and use the query-item co-occurrence information to make it more general. Overall the work is interesting and conference attendees would benefit from knowing about this work. Detailed comments are given in the respective pros and cons sections. Finally, a few suggestions regarding content that would add more value to the paper. - How quickly can recommendations be generated? - How much of it can be done online when the user is interacting with the system as opposed to offline? - Assuming you have enough offline processing power, how does the overhead compare with other methods? - How does the system and model evolve (is it incremental or exhaustive?) by incorporating new items, queries etc? ----------- Strengths and reasons to accept ----------- This is good work and the paper proposes a novel attempt at improving recommendations. + The paper is organized well and the relevant concepts are described clearly. The problem is relevant to personalized online recommendation engines. + There have been models that were quite successful that use content, user actions etc., and this paper adds a new relevant dimension to a relatively mature area, which is a strength. + The reasons for benefiting from choosing queries have been described succinctly - intent granularity, connections among interactions and intent boundaries. These are good insights and indicate the right reasons for pursuing such an approach. + Some thought has been given to practical issues like memory pressure due to vocabulary size, and the paper attempts to address it. Sections: 2.4: Sampled softmax to address size issue and multi-gpu embedding in order to parallelize are thoughtful additions. Other solutions didn’t have to handle this problem. 3.1: Experimental setting is relatively extensive. 3.3: Result on generalization ability of the model is persuasive. ----------- Weaknesses and limitations ----------- - Input sequence formulation (2.1) could have been shortened and Graph-based sequence augmentation could have been explained better (esp., the notation and the intuition behind the formulae 5 and 6), in its place. - 2.2: Initial state (edge set) of graph is specified E1 but it is not clear how the graph is evolved beyond the initial state (edge set) to get to E-alpha . More explanation is needed. For instance, it seems like E1 contains mostly 1-hop direct links as it is based on latest item-query links. So no common neighbors etc at this point. How does the augmentation bootstrap and proceed so that the threshold can be applied? ----------------------- REVIEW 3 --------------------- SUBMISSION: 6764 TITLE: Research: Query-Aware Sequential Recommendation AUTHORS: Zhankui He, Handong Zhao, Zhaowen Wang, Zhe Lin, Ajinkya Kale and Julian Mcauley ----------- Overall evaluation ----------- SCORE: 1 (weak accept) ----- TEXT: This paper describes a query-aware sequential recommendation approach by integrating query and item-interaction information into a graph-based recommendation approach. The authors describe their approach in a clear way and show that their approach is able to outperform different baselines. In order to improve the paper, the authors should give a short description of their plans for future work and should also include a statistical test (e.g., t-test) to show that their approach "really" outperforms the baselines. ----------- Strengths and reasons to accept ----------- - Clear description of approach in the paper - Strong emperical results ----------- Weaknesses and limitations ----------- - No future work given - No statistical tests