eSapiens's DEREK Module: Deep Extraction & Reasoning Engine for Knowledge with LLMs

• URL: http://arxiv.org/abs/2507.15863v1
• Date: Sun, 13 Jul 2025 05:54:01 GMT
• Title: eSapiens's DEREK Module: Deep Extraction & Reasoning Engine for Knowledge with LLMs
• Abstract summary: DEREK module ingests heterogeneous content (PDF, Office, web), splits it into 1,000-token overlapping chunks, and indexes them in a hybrid HNSW+BM25 store.<n>LangGraph verifier enforces citation overlap, regenerating answers until every claim is grounded.<n>All components run in containers, enforce end-to-end TLS 1.3 and AES-256.
• Score: 10.667949307405983
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present the DEREK (Deep Extraction & Reasoning Engine for Knowledge) Module, a secure and scalable Retrieval-Augmented Generation pipeline designed specifically for enterprise document question answering. Designed and implemented by eSapiens, the system ingests heterogeneous content (PDF, Office, web), splits it into 1,000-token overlapping chunks, and indexes them in a hybrid HNSW+BM25 store. User queries are refined by GPT-4o, retrieved via combined vector+BM25 search, reranked with Cohere, and answered by an LLM using CO-STAR prompt engineering. A LangGraph verifier enforces citation overlap, regenerating answers until every claim is grounded. On four LegalBench subsets, 1000-token chunks improve Recall@50 by approximately 1 pp and hybrid+rerank boosts Precision@10 by approximately 7 pp; the verifier raises TRACe Utilization above 0.50 and limits unsupported statements to less than 3%. All components run in containers, enforce end-to-end TLS 1.3 and AES-256. These results demonstrate that the DEREK module delivers accurate, traceable, and production-ready document QA with minimal operational overhead. The module is designed to meet enterprise demands for secure, auditable, and context-faithful retrieval, providing a reliable baseline for high-stakes domains such as legal and finance.

Related papers

• REIGN: Refurbished Embeddings with Integrated Guidance Networks for Efficient Context-Length Scaling [0.0]
  REIGN is a bi-encoder that operates on sequences of contextualised chunk embeddings from a frozen Guidance Network (GN) rather than on raw tokens.<n>We release a synthetic long-document retrieval benchmark for contrastive training and evaluation at long context lengths.<n>A paired significance test puts it on par with models 1.6-4.3x larger on the patent task, and it stays within 0.65 nDCG@10 of a 20x-larger model on LoCo.
  arXiv  Detail & Related papers  (2026-08-30T16:50:38Z)
• SA-Bench: Evaluating Semantic Alignment in LLM-Based Paper Reproduction [12.71956248550879]
  LLM agents can generate reproduction code, yet often produce scientifically unfaithful implementations.<n>We define this failure mode as semantic drift, where generated code silently diverges from the paper's specifications.<n>We introduce SemanticAlign-Bench, a diagnostic benchmark covering 30 papers from ICLR, ICML and NeurIPS 2025.
  arXiv  Detail & Related papers  (2026-08-25T08:48:48Z)
• An Interactive Agent for Requirement-Driven Candidate Sourcing [75.63902836826038]
  We present sys, the first requirements-driven candidate-sourcing agent.<n>It elicits, validates, retrieves, and verifies a vague people-request into a justified slate.<n>It dominates breadth ($100%$ coverage at $2.5times$ the yield) and is emphnear-orthogonal to the field.
  arXiv  Detail & Related papers  (2026-08-24T17:04:16Z)
• RAG-Based Auto-Configuration for Industrial Fieldbus Devices [2.623633900557428]
  Industrial device commissioning requires engineers to manually extract hundreds of protocol-specific parameters from PDF manuals.<n>This paper presents a production-oriented pipeline that automates device configuration end-to-end for OPC-UA, Profibus DP, and CANopen.
  arXiv  Detail & Related papers  (2026-08-09T10:07:13Z)
• VeriEvol: Scaling Multimodal Mathematical Reasoning via Verifiable Evol-Instruct [71.22358682581215]
  Scaling reinforcement learning for visual mathematical reasoning requires more than generating harder questions.<n>We instantiate this as VeriEvol, an iterative framework with two components.<n>On a five-benchmark visual-math suite, scaling evolved SFT data from 10K to 250K samples raises the mean accuracy from 35.42 to 54.73.
  arXiv  Detail & Related papers  (2026-06-22T16:17:30Z)
• From Attack Simulation to SIEM Rule: Deterministic Detection-as-Code Synthesis with Probe-Level Traceability [51.56484100374058]
  Security teams simulate attacks against their own systems to check whether their monitoring would catch a real intruder.<n>A human bridges that gap by hand, reading each finding and writing the corresponding Sigma rule.<n>We show this translation can be partially automated when probes are drawn from a locked corpus.
  arXiv  Detail & Related papers  (2026-06-03T14:26:25Z)
• Code-QA-Bench: Separating Code Reasoning from Documentation Memorization in Repository-Level QA [7.478327602641997]
  Code-QA-Bench is a framework for synthesizing repository-level code understanding benchmarks.<n>We generate 528 code-derivable and 100 doc-dependent tasks across 10 Python repositories.<n>The framework is open-source and applicable to any well-documented Python repository.
  arXiv  Detail & Related papers  (2026-05-28T02:52:58Z)
• Supporting System Testing with a Multi-Agent LLM-based Framework for Knowledge Graph Extraction: A Case Study with Ethernet Switch Systems [2.380943129168748]
  This paper focuses on Ethernet switch configuration manuals (ESCMs)<n>We propose a general framework that can be adapted to different industrial contexts.<n>We generate knowledge graphs (KGs) that capture configuration knowledge from ESCM in a structured form.
  arXiv  Detail & Related papers  (2026-05-18T23:05:21Z)
• Test-Time Strategies for More Efficient and Accurate Agentic RAG [58.44913384057518]
  Retrieval-Augmented Generation (RAG) systems face challenges with complex, multihop questions.<n>Such approaches can introduce inefficiencies, including repetitive retrieval of previously processed information.<n>We investigate test-time modifications to the Search-R1 pipeline to mitigate these identified shortcomings.
  arXiv  Detail & Related papers  (2026-03-12T19:18:59Z)
• Towards Global Retrieval Augmented Generation: A Benchmark for Corpus-Level Reasoning [50.27838512822097]
  We introduce GlobalQA, the first benchmark specifically designed to evaluate global RAG capabilities.<n>We propose GlobalRAG, a multi-tool collaborative framework that preserves structural coherence through chunk-level retrieval.<n>On the Qwen2.5-14B model, GlobalRAG achieves 6.63 F1 compared to the strongest baseline's 1.51 F1.
  arXiv  Detail & Related papers  (2025-10-30T07:29:14Z)
• DRBench: A Realistic Benchmark for Enterprise Deep Research [81.49694432639406]
  DRBench is a benchmark for evaluating AI agents on complex, open-ended deep research tasks in enterprise settings.<n>We release 15 deep research tasks across 10 domains, such as Sales, Cybersecurity, and Compliance.
  arXiv  Detail & Related papers  (2025-09-30T18:47:20Z)
• ODKE+: Ontology-Guided Open-Domain Knowledge Extraction with LLMs [1.904138488164197]
  ODKE+ is a production-grade system that automatically extracts and ingests millions of open-domain facts from web sources with high precision.<n>The system supports batch and streaming modes, processing over 9 million Wikipedia pages and ingesting 19 million high-confidence facts with 98.8% precision.
  arXiv  Detail & Related papers  (2025-09-04T23:05:23Z)
• DeAR: Dual-Stage Document Reranking with Reasoning Agents via LLM Distillation [26.719288314007002]
  Large Language Models (LLMs) have transformed listwise document reranking by enabling global reasoning over candidate sets.<n>We propose textbfDeeptextbfAgenttextbfRank (textbfDeAR), an open-source framework that decouples these tasks through a dual-stage approach.
  arXiv  Detail & Related papers  (2025-08-23T11:46:08Z)
• CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Reward [50.97588334916863]
  We develop CompassVerifier, an accurate and robust lightweight verifier model for evaluation and outcome reward.<n>It demonstrates multi-domain competency spanning math, knowledge, and diverse reasoning tasks, with the capability to process various answer types.<n>We introduce VerifierBench benchmark comprising model outputs collected from multiple data sources, augmented through manual analysis of metaerror patterns to enhance CompassVerifier.
  arXiv  Detail & Related papers  (2025-08-05T17:55:24Z)
• Towards Practical Defect-Focused Automated Code Review [8.370750734081088]
  We explore the full automation pipeline within the online recommendation service, analyzing industry-grade C++s.<n>We identify four key challenges: 1) capturing relevant context, 2) improving key inclusion, 3) reducing false alarm rates (FAR), and 4) integrating human bug slicing.<n>Our approach, validated on real-world merge requests from historical fault reports, achieves a 2x improvement over standard LLMs and a 10x gain over previous baselines.
  arXiv  Detail & Related papers  (2025-05-23T14:06:26Z)
• AGENTIF: Benchmarking Instruction Following of Large Language Models in Agentic Scenarios [51.46347732659174]
  Large Language Models (LLMs) have demonstrated advanced capabilities in real-world agentic applications.<n>AgentIF is the first benchmark for systematically evaluating LLM instruction following ability in agentic scenarios.
  arXiv  Detail & Related papers  (2025-05-22T17:31:10Z)
• Document Attribution: Examining Citation Relationships using Large Language Models [62.46146670035751]
  We propose a zero-shot approach that frames attribution as a straightforward textual entailment task.<n>We also explore the role of the attention mechanism in enhancing the attribution process.
  arXiv  Detail & Related papers  (2025-05-09T04:40:11Z)
• Enhancing Domain-Specific Retrieval-Augmented Generation: Synthetic Data Generation and Evaluation using Reasoning Models [0.6827423171182154]
  Retrieval-Augmented Generation (RAG) systems face significant performance gaps when applied to technical domains.<n>We propose a framework combining granular evaluation metrics with synthetic data generation to optimize domain-specific RAG performance.<n>Our empirical analysis reveals critical insights: smaller chunks (less than 10 tokens) improve precision by 31-42%.
  arXiv  Detail & Related papers  (2025-02-21T06:38:57Z)
• From Documents to Dialogue: Building KG-RAG Enhanced AI Assistants [28.149173430599525]
  We use a Retrieval-Augmented Generation (RAG) framework powered by a Knowledge Graph (KG) to retrieve relevant information from external knowledge sources.<n>Our KG-RAG system retrieves relevant provenances, which are added to the user prompts context before being sent to the LLM generating the response.<n>Our evaluation demonstrates that this approach significantly enhances response relevance, reducing irrelevant answers by over 50% and increasing fully relevant answers by 88% compared to the existing production system.
  arXiv  Detail & Related papers  (2025-02-21T06:22:12Z)
• Towards Copyright Protection for Knowledge Bases of Retrieval-augmented Language Models via Reasoning [58.57194301645823]
  Large language models (LLMs) are increasingly integrated into real-world personalized applications.<n>The valuable and often proprietary nature of the knowledge bases used in RAG introduces the risk of unauthorized usage by adversaries.<n>Existing methods that can be generalized as watermarking techniques to protect these knowledge bases typically involve poisoning or backdoor attacks.<n>We propose name for harmless' copyright protection of knowledge bases.
  arXiv  Detail & Related papers  (2025-02-10T09:15:56Z)
• Unsupervised Text Representation Learning via Instruction-Tuning for Zero-Shot Dense Retrieval [19.422003299376]
  We introduce a novel unsupervised text representation learning technique via instruction-tuning. We demonstrate the corpus representation can be augmented by the representations of relevant synthetic queries. We significantly improve the average zero-shot retrieval performance on all metrics.
  arXiv  Detail & Related papers  (2024-09-24T23:03:13Z)
• ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code [76.84199699772903]
  ML-Bench is a benchmark rooted in real-world programming applications that leverage existing code repositories to perform tasks. To evaluate both Large Language Models (LLMs) and AI agents, two setups are employed: ML-LLM-Bench for assessing LLMs' text-to-code conversion within a predefined deployment environment, and ML-Agent-Bench for testing autonomous agents in an end-to-end task execution within a Linux sandbox environment.
  arXiv  Detail & Related papers  (2023-11-16T12:03:21Z)
• BooookScore: A systematic exploration of book-length summarization in the era of LLMs [53.42917858142565]
  We develop an automatic metric, BooookScore, that measures the proportion of sentences in a summary that do not contain any of the identified error types. We find that closed-source LLMs such as GPT-4 and 2 produce summaries with higher BooookScore than those generated by open-source models.
  arXiv  Detail & Related papers  (2023-10-01T20:46:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.