$\mathcal{S}^2$IT: Stepwise Syntax Integration Tuning for Large Language Models in Aspect Sentiment Quad Prediction

• URL: http://arxiv.org/abs/2604.23296v1
• Date: Sat, 25 Apr 2026 13:24:29 GMT
• Title: $\mathcal{S}^2$IT: Stepwise Syntax Integration Tuning for Large Language Models in Aspect Sentiment Quad Prediction
• Abstract summary: syntactic structure information is underutilized in the generative paradigm of large language models (LLMs)<n>S2IT progressively integrates syntactic structure knowledge into LLMs through a multi-step tuning process.<n> Experiments demonstrate that S2IT significantly improves state-of-the-art performance across multiple datasets.
• Score: 51.165432266846096
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Aspect Sentiment Quad Prediction (ASQP) has seen significant advancements, largely driven by the powerful semantic understanding and generative capabilities of large language models (LLMs). However, while syntactic structure information has been proven effective in previous extractive paradigms, it remains underutilized in the generative paradigm of LLMs due to their limited reasoning capabilities. In this paper, we propose S^2IT, a novel Stepwise Syntax Integration Tuning framework that progressively integrates syntactic structure knowledge into LLMs through a multi-step tuning process. The training process is divided into three steps. S^2IT decomposes the quadruple generation task into two stages: 1) Global Syntax-guided Extraction and 2) Local Syntax-guided Classification, integrating both global and local syntactic structure information. Finally, Fine-grained Structural Tuning enhances the model's understanding of syntactic structures through the prediction of element links and node classification. Experiments demonstrate that S^2IT significantly improves state-of-the-art performance across multiple datasets. Our implementation will be open-sourced at https://github.com/DMIRLAB-Group/S2IT.

Related papers

• Weave of Formal Thought [51.56484100374058]
  We introduce Weave of Formal Thought (WoFT), a paradigm uniting rigorous syntactic validation with learned structural representations.<n>We present a latent-variable fine-tuning method training the language model to interleave non-terminal grammar symbols directly into generation.<n>For Python, fine-tuning StarCoder2-3B with our RWS objective reduces per-token cross-entropy by 14.3% relative to a text-only SFT baseline.
  arXiv  Detail & Related papers  (2026-06-24T15:58:11Z)
• Structured Prompt Optimization Meets Reinforcement Learning for Global and Local Interpretability over Complex Text [14.95562040863983]
  LLMs have advanced text classification, yet existing paradigms face a trade-off.<n>We introduce eXTC with three progressive stages: learning a SOP, reasoning distillation from a large teacher LLM into a compact LM, and expanding reasoning capabilities beyond the initial SOP.<n>This design enables (i) fast inference via a compact LM, with (ii) inference-time local reasoning traces, alongside a global, modular explanation of its learned domain rules, while (iii) significantly outperforming existing paradigms across diverse benchmarks in both classification performance and explanation quality, with stage-by-stage gains.
  arXiv  Detail & Related papers  (2026-05-27T20:29:41Z)
• Improving LLM Reasoning with Homophily-aware Structural and Semantic Text-Attributed Graph Compression [55.51959317490934]
  Large language models (LLMs) have demonstrated promising capabilities in Text-Attributed Graph (TAG) understanding.<n>We argue that graphs inherently contain rich structural and semantic information, and that their effective exploitation can unlock potential gains in LLMs reasoning performance.<n>We propose Homophily-aware Structural and Semantic Compression for LLMs (HS2C), a framework centered on exploiting graph homophily.
  arXiv  Detail & Related papers  (2026-01-13T03:35:18Z)
• STELLA: Guiding Large Language Models for Time Series Forecasting with Semantic Abstractions [4.169671705130711]
  We propose STELLA, a framework that mines and injects structured supplementary and complementary information.<n>StELLA employs a dynamic semantic abstraction mechanism that decouples input series into trend, seasonality, and residual components.<n>Experiments on eight benchmark datasets demonstrate that STELLA outperforms state-of-the-art methods in long- and short-term forecasting.
  arXiv  Detail & Related papers  (2025-12-04T14:56:36Z)
• TIT: A Tree-Structured Instruction Tuning Approach for LLM-Based Code Translation [11.882496324328905]
  We propose TIT, a Tree-structured Instruction Tuning paradigm for LLM-based code translation.<n>To mitigate syntactic confusion, the syntactic information representation module integrates language-agnostic syntactic features.<n>To generate high-quality fine-grained parallel data, the fine-grained parallel dataset augmentation module aligns nodes with code segments.
  arXiv  Detail & Related papers  (2025-10-10T13:53:46Z)
• CoT Referring: Improving Referring Expression Tasks with Grounded Reasoning [67.18702329644526]
  CoT Referring enhances model reasoning across modalities through a structured, chain-of-thought training data structure.<n>We restructure the training data to enforce a new output form, providing new annotations for existing datasets.<n>We also integrate detection and segmentation capabilities into a unified MLLM framework, training it with a novel adaptive weighted loss to optimize performance.
  arXiv  Detail & Related papers  (2025-10-03T08:50:21Z)
• The Unreasonable Effectiveness of Model Merging for Cross-Lingual Transfer in LLMs [45.08958917457921]
  Large language models (LLMs) still struggle across tasks outside of high-resource languages.<n>In this work, we investigate cross-lingual transfer to lower-resource languages where task-specific post-training data is scarce.
  arXiv  Detail & Related papers  (2025-05-23T20:28:31Z)
• Contextualization Distillation from Large Language Model for Knowledge Graph Completion [51.126166442122546]
  We introduce the Contextualization Distillation strategy, a plug-in-and-play approach compatible with both discriminative and generative KGC frameworks. Our method begins by instructing large language models to transform compact, structural triplets into context-rich segments. Comprehensive evaluations across diverse datasets and KGC techniques highlight the efficacy and adaptability of our approach.
  arXiv  Detail & Related papers  (2024-01-28T08:56:49Z)
• kNN-ICL: Compositional Task-Oriented Parsing Generalization with Nearest Neighbor In-Context Learning [50.40636157214161]
  Task-Oriented Parsing (TOP) enables conversational assistants to interpret user commands expressed in natural language. LLMs have achieved impressive performance in computer programs based on a natural language prompt. This paper focuses on harnessing the capabilities of LLMs for semantic parsing tasks.
  arXiv  Detail & Related papers  (2023-12-17T17:26:50Z)
• Leveraging Code to Improve In-context Learning for Semantic Parsing [48.66031267718704]
  In-context learning (ICL) is an appealing approach for semantic parsing due to its few-shot nature and improved generalization. We improve the effectiveness of ICL for semantic parsing by (1) using general-purpose programming languages such as Python instead of DSLs, and (2) augmenting prompts with a structured domain description.
  arXiv  Detail & Related papers  (2023-11-16T02:50:06Z)
• Syntax-Aware Complex-Valued Neural Machine Translation [14.772317918560548]
  We propose a method to incorporate syntax information into a complex-valued-Decoder architecture. The proposed model jointly learns word-level and syntax-level attention scores from the source side to the target side using an attention mechanism. The experimental results demonstrate that the proposed method can bring significant improvements in BLEU scores on two datasets.
  arXiv  Detail & Related papers  (2023-07-17T15:58:05Z)
• SIT3: Code Summarization with Structure-Induced Transformer [48.000063280183376]
  We propose a novel model based on structure-induced self-attention, which encodes sequential inputs with highly-effective structure modeling. Our newly-proposed model achieves new state-of-the-art results on popular benchmarks.
  arXiv  Detail & Related papers  (2020-12-29T11:37:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.