Reviewer #2
Questions
1. Brief summary of the paper
This paper tackles sequential sentence classification by proposing SeqGNN, a hierarchical model that combines SciBERT embeddings with a graph neural network over sentence and word nodes. The method builds a graph linking all [SEP] sentence markers and their corresponding word tokens, enabling the GNN to capture higher-order dependencies beyond BERT’s sequential context. After applying multiple GraphConv layers with residual connections and normalization, an MLP predicts each sentence label. Experiments on five datasets show that SeqGNN outperforms strong baselines, with ablations confirming the value of the graph structure and GNN layers.
2. List three, or more, strong aspects of this paper. Please number each point.
1. The model’s integration of SciBERT with a graph over sentence and word nodes is innovative and enables interactions that standard Transformers may miss.

2. SeqGNN achieves consistently higher micro-F1 scores than strong baselines across all five datasets, demonstrating clear empirical advantages.

3. The paper includes thorough ablation studies showing the contribution of graph edges, multiple GNN layers, and residual connections to performance gains.

4. The architecture and methodology are clearly explained with helpful figures, equations, and well-organized sections.

5. Use of public datasets and planned code release improve reproducibility and confidence in the experimental results.
3. List three, or more, weak aspects of this paper. Please number each point.
1. The combined SciBERT–GNN architecture may be slow and resource-heavy, and the paper provides no runtime or efficiency measurements to assess scalability.

2. The graph construction details are unclear, especially regarding whether [SEP] nodes form a clique and how word nodes connect, making the topology somewhat ambiguous.

3. The model’s novelty is incremental since adding structural layers to BERT is common, and the paper could better distinguish SeqGNN from prior GCN/HGNN approaches.

4. Evaluation is limited to abstract-style datasets with only micro-F1 reported, leaving uncertainty about performance on longer documents or varied label distributions.

5. A simpler baseline—such as BERT followed by a BiLSTM or CRF—would help isolate the specific benefit of the GNN component compared to traditional sequential models.
4. Detailed comments to the authors.
1. The graph idea is strong, but the paper should explicitly state the edge-construction rules in clear bullet form to remove ambiguity.

2. Since SeqGNN performs similarly to baselines on CODA-19, discussing what makes that dataset different would clarify when the graph helps most.

3. The ablation figures are useful, but a brief explanation of oversmoothing and clearer captions for Fig.2(b) and (c) would aid reader understanding.

4. Providing model size, runtime, and overhead compared to plain SciBERT would help assess the practical cost of adding GNN layers.

5. Notation around [SEP] tokens should be made consistent, and minor typos and reference formatting issues should be corrected for clarity.
5. Overall Recommendation
Weak Accept: Borderline paper, tending to accept
Reviewer #3
Questions
1. Brief summary of the paper
This paper proposes SeqGNN, a hierarchical model that integrates pretrained language models with graph neural networks for sequential sentence classification. Sentences are represented by contextualized embeddings from SciBERT, and a graph is constructed over sentence-level [SEP] tokens and word tokens to explicitly model structural dependencies between sentences. Graph neural network layers are applied to capture higher-order sentence dependencies, with residual connections used to preserve contextual information. Extensive experiments on five benchmark datasets from different domains demonstrate that SeqGNN consistently outperforms strong BERT-based and graph-based baselines.
2. List three, or more, strong aspects of this paper. Please number each point.
(1)Clear motivation and problem formulation: The paper clearly identifies a limitation of existing BERT-based SSC models, namely the lack of explicit modeling of sentence-level structural dependencies.
(2)Well-designed integration of language models and GNNs: The construction of a sentence/word graph and the application of GNN layers on top of contextualized embeddings is technically sound and well justified.
(3)Consistent empirical improvements across datasets: SeqGNN achieves stable and statistically significant improvements over strong baselines on multiple benchmark datasets from different domains.
(4)Comprehensive ablation studies: The paper includes thorough ablations on graph structure, number of GNN layers, residual connections, and GNN operators, which strengthens the empirical credibility of the approach.
3. List three, or more, weak aspects of this paper. Please number each point.
(1)Incremental methodological novelty: While effective, the core idea of combining pretrained language models with GNNs has been explored in prior work, and the novelty mainly lies in the specific graph construction and integration strategy.
(2)Graph construction heuristics are fixed: The sentence/word graph structure is manually designed and task-agnostic, without learning or adapting the structure from data.
(3)Limited analysis of model complexity and scalability: The paper does not deeply discuss memory usage, inference latency, or scalability to very long documents.
(4)Evaluation metric choice is limited: Only micro-F1 is reported; macro-F1 or label-level imbalance analysis could provide additional insight.
4. Detailed comments to the authors.
This paper presents a solid and well-executed approach to sequential sentence classification by explicitly modeling sentence-level structural dependencies using graph neural networks. The integration of SciBERT with GNN layers is technically sound, and the empirical results consistently demonstrate performance gains across multiple datasets.
However, the contribution is largely incremental in nature, as the general idea of combining language models and graph neural networks has been explored before. The manually designed graph structure, while intuitive, may limit flexibility and adaptability across domains. In addition, the paper would benefit from a deeper discussion of computational efficiency and scalability, especially for longer documents.
5. Overall Recommendation
Weak Accept: Borderline paper, tending to accept
Reviewer #4
Questions
1. Brief summary of the paper
This paper studies the task of Sequential Sentence Classification, proposing a hierarchical model that integrates pre-trained language models (SciBERT) with graph neural networks (GNNs).

The proposed approach is evaluated on five real-world datasets from multiple domains and shows consistent improvements over strong baselines, including Sequential BERT and CRF-based hierarchical models.
2. List three, or more, strong aspects of this paper. Please number each point.
- The paper clearly describes the SSC task and provides a reasonable motivation for exploring structural sentence dependencies beyond purely sequential modeling.

- Integrating SciBERT with GNN layers is a sensible design choice, and the architecture is described in sufficient detail to be reproducible.

- The method is evaluated on multiple datasets from different domains, with comparisons against several strong baselines. Ablation studies on graph structure, number of GNN layers, and residual connections are provided.
3. List three, or more, weak aspects of this paper. Please number each point.
- The proposed solution combines existing components (SciBERT, GNNs, residual connections) in a relatively straightforward manner. The overall contribution does not introduce fundamentally new modeling ideas for SSC.

- The construction of the sentence/token graph relies on hand-crafted design choices (e.g., connecting [SEP] tokens, sequential vs fully connected graphs). The paper provides limited justification for these choices and does not explore whether the graph structure could be learned or adapted to different document types.

- It is not fully clear whether the observed improvements stem from explicit modeling of higher-order sentence dependencies or simply from adding additional modeling capacity on top of SciBERT. Deeper ablation studies are highly recommended

- Limited discussion of scalability and generalization. The experiments focus on relatively short documents (abstracts). Since the graph includes all tokens and sentence nodes, scalability to longer documents is unclear and not discussed in depth.
4. Detailed comments to the authors.
The paper explores a reasonable direction for improving sequential sentence classification by incorporating graph neural networks to model sentence dependencies. While the approach is technically sound and the experiments are carefully conducted, the overall contribution appears incremental. The proposed model mainly combines existing techniques, and the novelty lies more in architectural composition than in new methodological insights.

The graph construction is largely heuristic, and a deeper analysis of its necessity, robustness, or adaptability would strengthen the paper. Additionally, clarifying whether the gains primarily come from explicit structural modeling or increased representational capacity would help better assess the contribution. A discussion of scalability to longer documents and more complex settings would also be valuable.

Overall, while the paper demonstrates some empirical improvements, the limited novelty and reliance on manually designed components reduce its impact.
5. Overall Recommendation
Weak Accept: Borderline paper, tending to accept