GPT-5 vs Other LLMs in Long Short-Context Performance

• URL: http://arxiv.org/abs/2602.14188v1
• Date: Sun, 15 Feb 2026 15:26:25 GMT
• Title: GPT-5 vs Other LLMs in Long Short-Context Performance
• Authors: Nima Esmi, Maryam Nezhad-Moghaddam, Fatemeh Borhani, Asadollah Shahbahrami, Amin Daemdoost, Georgi Gaydadjiev,
• Abstract summary: This paper evaluates the performance of four state-of-the-art models (Grok-4, GPT-4, Gemini 2.5, and GPT-5) on long short-context tasks.<n>As the input volume on the social media dataset exceeds 5K posts (70K tokens), the performance of all models degrades significantly.<n>In the GPT-5 model, despite the sharp decline in accuracy, its precision remained high at approximately 95%.
• Score: 2.640490999540592
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the significant expansion of the context window in Large Language Models (LLMs), these models are theoretically capable of processing millions of tokens in a single pass. However, research indicates a significant gap between this theoretical capacity and the practical ability of models to robustly utilize information within long contexts, especially in tasks that require a comprehensive understanding of numerous details. This paper evaluates the performance of four state-of-the-art models (Grok-4, GPT-4, Gemini 2.5, and GPT-5) on long short-context tasks. For this purpose, three datasets were used: two supplementary datasets for retrieving culinary recipes and math problems, and a primary dataset of 20K social media posts for depression detection. The results show that as the input volume on the social media dataset exceeds 5K posts (70K tokens), the performance of all models degrades significantly, with accuracy dropping to around 50-53% for 20K posts. Notably, in the GPT-5 model, despite the sharp decline in accuracy, its precision remained high at approximately 95%, a feature that could be highly effective for sensitive applications like depression detection. This research also indicates that the "lost in the middle" problem has been largely resolved in newer models. This study emphasizes the gap between the theoretical capacity and the actual performance of models on complex, high-volume data tasks and highlights the importance of metrics beyond simple accuracy for practical applications.

Related papers

• RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension [65.81339691942757]
  RPC-Bench is a large-scale question-answering benchmark built from review-rebuttal exchanges of high-quality computer science papers.<n>We design a fine-grained taxonomy aligned with the scientific research flow to assess models' ability to understand and answer why, what, and how questions in scholarly contexts.
  arXiv  Detail & Related papers  (2026-01-14T11:37:00Z)
• Information Capacity: Evaluating the Efficiency of Large Language Models via Text Compression [53.39128997308138]
  We introduce information capacity, a measure of model efficiency based on text compression performance.<n> Empirical evaluations on mainstream open-source models show that models of varying sizes within a series exhibit consistent information capacity.<n>A distinctive feature of information capacity is that it incorporates tokenizer efficiency, which affects both input and output token counts.
  arXiv  Detail & Related papers  (2025-11-11T10:07:32Z)
• Klear-Reasoner: Advancing Reasoning Capability via Gradient-Preserving Clipping Policy Optimization [17.149024413701014]
  We present Klear-Reasoner, a model with long reasoning capabilities that demonstrates careful deliberation during problem solving.<n>This report provides an in-depth analysis of the reasoning model, covering the entire post-training workflow.
  arXiv  Detail & Related papers  (2025-08-11T05:17:51Z)
• EfficientLLM: Efficiency in Large Language Models [64.3537131208038]
  Large Language Models (LLMs) have driven significant progress, yet their growing counts and context windows incur prohibitive compute, energy, and monetary costs.<n>We introduce EfficientLLM, a novel benchmark and the first comprehensive empirical study evaluating efficiency techniques for LLMs at scale.
  arXiv  Detail & Related papers  (2025-05-20T02:27:08Z)
• ReasoningV: Efficient Verilog Code Generation with Adaptive Hybrid Reasoning Model [7.798551697095774]
  ReasoningV is a novel model that integrates trained intrinsic capabilities with dynamic inference adaptation for Verilog code generation.<n>Our framework introduces three complementary innovations: ReasoningV-5K, a high-quality dataset of 5,000 functionally verified instances with reasoning paths created through multi-dimensional filtering of PyraNet samples.<n> Experimental results demonstrate ReasoningV's effectiveness with a pass@1 accuracy of 57.8% on VerilogEval-human.
  arXiv  Detail & Related papers  (2025-04-20T10:16:59Z)
• An Empirical Comparison of Text Summarization: A Multi-Dimensional Evaluation of Large Language Models [2.1945750784330067]
  This research evaluates summarization performance across 17 large language models (OpenAI, Google, Anthropic, open-source)<n>We assessed models on seven diverse datasets using metrics for factual consistency, semantic similarity, lexical overlap, and human-like quality.
  arXiv  Detail & Related papers  (2025-04-06T16:24:22Z)
• Benchmarking Reasoning Robustness in Large Language Models [76.79744000300363]
  We find significant performance degradation on novel or incomplete data.<n>These findings highlight the reliance on recall over rigorous logical inference.<n>This paper introduces a novel benchmark, termed as Math-RoB, that exploits hallucinations triggered by missing information to expose reasoning gaps.
  arXiv  Detail & Related papers  (2025-03-06T15:36:06Z)
• Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capability [53.51560766150442]
  Critical tokens are elements within reasoning trajectories that significantly influence incorrect outcomes.<n>We present a novel framework for identifying these tokens through rollout sampling.<n>We show that identifying and replacing critical tokens significantly improves model accuracy.
  arXiv  Detail & Related papers  (2024-11-29T18:58:22Z)
• Evaluating the Impact of Compression Techniques on Task-Specific Performance of Large Language Models [0.0]
  Large language models (LLMs) offer powerful capabilities but incur substantial computational costs. This study evaluates the impact of popular compression methods on the LLaMA-2-7B model. We show that while SparseGPT and Wanda preserve perplexity even at 50% sparsity, they suffer significant degradation on downstream tasks.
  arXiv  Detail & Related papers  (2024-09-17T14:34:11Z)
• Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning [110.80663974060624]
  Key-Point-Driven Data Synthesis (KPDDS) is a novel data synthesis framework that synthesizes question-answer pairs. KPDDS ensures the generation of novel questions with rigorous quality control and substantial scalability. We present KPMath, an extensive synthetic dataset tailored for mathematical reasoning, comprising over 800K question-answer pairs.
  arXiv  Detail & Related papers  (2024-03-04T18:58:30Z)
• Automated Root Causing of Cloud Incidents using In-Context Learning with GPT-4 [23.856839017006386]
  Root Cause Analysis (RCA) plays a pivotal role in the incident diagnosis process for cloud services. GPT-4 model's immense size presents challenges when trying to fine-tune it on user data. We propose an in-context learning approach for automated root causing, which eliminates the need for fine-tuning.
  arXiv  Detail & Related papers  (2024-01-24T21:02:07Z)
• GPT-Neo for commonsense reasoning -- a theoretical and practical lens [0.46040036610482665]
  We evaluate the performance of the GPT-neo model using $6$ commonsense reasoning benchmark tasks. We aim to examine the performance of smaller models using the GPT-neo models against several larger model baselines.
  arXiv  Detail & Related papers  (2022-11-28T17:49:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.