------------------------- METAREVIEW ------------------------ Thank you for submitting your work to SIGIR 2024. All reviewers find the work interesting and are positive of the contributions. Nevertheless, as points of improvements, they have raised several concerns in their reviews. Please take these comments into account when preparing the camera-ready version of your paper so as to further improve the paper. ----------------------- REVIEW 1 --------------------- SUBMISSION: 1209 TITLE: Fair Sequential Recommendation without User Demographics AUTHORS: Huimin Zeng, Zhankui He, Zhenrui Yue, Julian McAuley and Dong Wang ----------- Relevance to SIGIR ----------- SCORE: 4 (good) ----------- Originality of the work ----------- SCORE: 4 (good) ----- TEXT: Fairness is an important problem in recommender systems. ----------- Technical soundness ----------- SCORE: 4 (good) ----- TEXT: The authors design dirichlet neighbor smoothing and worst-case performance optimization to improve fairness. ----------- Quality of presentation ----------- SCORE: 4 (good) ----- TEXT: This work is very well-written ----------- Related work ----------- SCORE: 4 (good) ----------- Perceived significance or potential impact of the contribution ----------- SCORE: 4 (good) ----------- Strengths ----------- - Fairness is an important problem in recommender systems. - The authors design Dirichlet Neighbor Smoothing and Worst-case Performance Optimization to improve fairness. - Extensive experiments were conducted on three real-world datasets. ----------- Weaknesses ----------- - The authors focus on fair sequential recommendation. However, the authors do not distinguish the difference between fair sequential recommendation and fair traditional recommendation. Thus, this paper cannot explain the particularity of fair sequential recommendation, and so the motivation and challenge are unclear. - The group fairness metric defined by the authors is not related to sequential recommendation. Therefore, using this fairness metric to measure the results of sequential recommendation is not convincing. ----------- Overall recommendation ----------- SCORE: 1 (weak accept) ----------- Detailed comments to authors ----------- Please refer to the weaknesses. ----------------------- REVIEW 2 --------------------- SUBMISSION: 1209 TITLE: Fair Sequential Recommendation without User Demographics AUTHORS: Huimin Zeng, Zhankui He, Zhenrui Yue, Julian McAuley and Dong Wang ----------- Relevance to SIGIR ----------- SCORE: 5 (excellent) ----------- Originality of the work ----------- SCORE: 3 (fair) ----- TEXT: The originality of this work is fair. ----------- Technical soundness ----------- SCORE: 3 (fair) ----- TEXT: The technical soundness is fair. ----------- Quality of presentation ----------- SCORE: 4 (good) ----- TEXT: The presentation is clear. ----------- Related work ----------- SCORE: 4 (good) ----- TEXT: The literature review is good enough. ----------- Perceived significance or potential impact of the contribution ----------- SCORE: 4 (good) ----- TEXT: The potential impact might be high. ----------- Strengths ----------- S1. The paper under review focuses on the fairness of sequential models without explicitly relying on demographic attributes, which is a significant strength. This approach to fairness is innovative and could lead to a few new insights and methodologies in the field. S2. The paper introduces a model-agnostic and demographic-agnostic debiasing framework, which is a significant advantage. This framework enhances the generalizability of the proposed approach and can be applied across various sequential models and datasets. S3. The approach of identifying potential demographic attributes and under-represented groups by closely monitoring the magnitude of gradient updates is innovative. ----------- Weaknesses ----------- W1. While the Dirichlet Neighbor Smoothing method is a valid approach to fairness in sequential models, its reliance on embedding sharing may limit its novelty and originality. The use of embedding sharing in this method is relatively common in the field, which may limit its novelty and originality. By integrating more advanced graph methods into the methodology, the paper could enhance the innovation and effectiveness of its approach to fairness in sequential modeling. W2. A more in-depth discussion could delve into the specific patterns or trends observed in gradient updates that are indicative of biases related to demographic attributes or under-represented groups. W3. The impact of removing DNS and DRO modules on model performance and fairness metrics should be elucidated to provide a comprehensive understanding of their contributions to the proposed methodology. ----------- Overall recommendation ----------- SCORE: 1 (weak accept) ----------- Detailed comments to authors ----------- The paper presents a novel approach to addressing fairness concerns in sequential models by monitoring gradient updates to detect potential biases related to demographic attributes and under-represented groups. However, the paper's proposed Dirichlet Neighbor Smoothing method may lack novelty in its use of embedding sharing, which is a relatively common technique in the field. Furthermore, the paper could benefit from a more detailed discussion on the relationship between gradient updates and demographic attributes or under-represented groups. Providing a more detailed explanation of the performance changes observed in the ablation experiments would enhance the paper's clarity. The paper offers insights into fairness in sequential models through its innovative approach to bias detection and mitigation. While the proposed method shows promise in addressing fairness concerns, further exploration and elaboration on certain aspects could enhance the paper's contribution to the field of fairness in machine learning algorithms. By addressing the identified weaknesses, the paper has the potential to inspire future research and advancements in the pursuit of fair and unbiased sequential models. ----------------------- REVIEW 3 --------------------- SUBMISSION: 1209 TITLE: Fair Sequential Recommendation without User Demographics AUTHORS: Huimin Zeng, Zhankui He, Zhenrui Yue, Julian McAuley and Dong Wang ----------- Relevance to SIGIR ----------- SCORE: 5 (excellent) ----------- Originality of the work ----------- SCORE: 5 (excellent) ----- TEXT: The work is original ----------- Technical soundness ----------- SCORE: 5 (excellent) ----- TEXT: Paper is technically sound. ----------- Quality of presentation ----------- SCORE: 4 (good) ----------- Related work ----------- SCORE: 4 (good) ----------- Perceived significance or potential impact of the contribution ----------- SCORE: 4 (good) ----------- Strengths ----------- 1. This paper proposes a novel framework (Agnostic FairSeqRec, A-FSR) that enables fair sequential recommendations without relying on users' demographic information. Moreover, A-FSR is a model-agnostic framework, meaning it can be applied to any existing sequential recommendation model, significantly broadening its practicality and scope of application. 2. Through experiments on real-world datasets, the paper demonstrates A-FSR's effectiveness in improving group fairness and shows performance gains over other state-of-the-art baseline methods. ----------- Weaknesses ----------- Minor issue: the paper lacks the dataset statistics. ----------- Overall recommendation ----------- SCORE: 2 (accept) ----------- Detailed comments to authors ----------- The paper addresses the challenge of building a fair sequential recommendation system without relying on users' demographic attributes, such as gender. Much of the existing literature on fair recommendations develops methods based on users' demographic information. However, in real-world scenarios, users may be reluctant to share their demographic information with systems due to privacy concerns and convenience, which limits the applicability of many existing methods. Additionally, while sequential recommendation (SR) models have achieved state-of-the-art performance compared to traditional collaborative filtering (CF) recommenders and can represent users solely based on user-item interactions (user-free), this creates a misconception that SR models are inherently free from group unfairness.To tackle this issue, the paper proposes Agnostic FairSeqRec (A-FSR), a model-agnostic and demographic-agnostic debiasing framework for sequential recommendation that does not require user s' demographic attributes. A-FSR achieves its goal through the following two main steps: 1. Reducing the correlation between potential stereotypical patterns in the input sequences and the final recommendations via Dirichlet neighbor smoothing. 2. Estimating an under-represented group of sequences using a gradient-based heuristic and implicitly shifting the training focus towards this under-represented group by minimizing a distributionally robust optimization (DRO) based objective. Overall, this article raises an interesting and practically significant issue of fairness for SR. The model is well-designed and analyzes Dirichlet neighbor smoothing as well as performance optimization in the worst-case scenario. The experimental results of the paper demonstrate that A-FSR significantly improves group fairness in sequential recommendation while outperforming other state-of-the-art baselines in terms of performance. ----------------------- REVIEW 4 --------------------- SUBMISSION: 1209 TITLE: Fair Sequential Recommendation without User Demographics AUTHORS: Huimin Zeng, Zhankui He, Zhenrui Yue, Julian McAuley and Dong Wang ----------- Relevance to SIGIR ----------- SCORE: 4 (good) ----------- Originality of the work ----------- SCORE: 3 (fair) ----- TEXT: The paper focuses on a relatively new task and investigates how to build fair recommendation methods for sequential recommender systems without user demographic information. ----------- Technical soundness ----------- SCORE: 3 (fair) ----- TEXT: 1. The compatibility and effectiveness of the two fairness modules in this paper need to be further verified. 2. It would be better to add more theoretical analyses. ----------- Quality of presentation ----------- SCORE: 3 (fair) ----------- Related work ----------- SCORE: 2 (poor) ----------- Perceived significance or potential impact of the contribution ----------- SCORE: 3 (fair) ----------- Strengths ----------- 1. A general debiasing framework is proposed and it can be applied to sequential recommender systems in the absence of user demographic information. 2. The authors propose two novel modules, Dirichlet Neighbourhood Smoothing and Worst-Case Performance Optimization. The former blurs the correlation between the stereotypical patterns and predictions. The latter enhances the robustness of the fairness improvements on potential under-represented users. ----------- Weaknesses ----------- 1. In the ablation study, Table 5 was not compared with the original model, which is not easy to read. In addition, on the ML-100K dataset on the BERT4Rec model, I find that the two modules DNS and DRO do not improve the fairness, while the combination of the two achieves the fairness improvement, what is the reason for this phenomenon? 2. The authors direct use the DNS and DRO modules to cluster information based on the magnitude of the loss. It would be better to further explain the rationale of the decision method and explore whether the division can be applied to other cluster-based models. 3. The figure 3 in the paper lacks information. 4. Some important related works are missing, such as [1,2]. It would be better to comprehensively research related work in this field and make a comparison with them. å\ [1] Fairness without Demographics through Adversarially Reweighted Learning [2] Learning from Failure: Training Debiased Classifier from Biased Classifier ----------- Overall recommendation ----------- SCORE: 1 (weak accept) ----------- Detailed comments to authors ----------- Pros: 1. A general debiasing framework is proposed and it can be applied to sequential recommender systems in the absence of user demographic information. 2. The authors propose two novel modules, Dirichlet Neighbourhood Smoothing and Worst-Case Performance Optimization. The former blurs the correlation between the stereotypical patterns and predictions. The latter enhances the robustness of the fairness improvements on potential under-represented users. Cons: 1. In the ablation study, Table 5 was not compared with the original model, which is not easy to read. In addition, on the ML-100K dataset on the BERT4Rec model, I find that the two modules DNS and DRO do not improve the fairness, while the combination of the two achieves the fairness improvement, what is the reason for this phenomenon? 2. The authors direct use the DNS and DRO modules to cluster information based on the magnitude of the loss. It would be better to further explain the rationale of the decision method and explore whether the division can be applied to other cluster-based models. 3. The figure 3 in the paper lacks information. 4. Some important related works are missing, such as [1,2]. It would be better to comprehensively research related work in this field and make a comparison with them. å\ [1] Fairness without Demographics through Adversarially Reweighted Learning [2] Learning from Failure: Training Debiased Classifier from Biased Classifier