P-ICL: Point In-Context Learning for Named Entity Recognition with Large Language Models

• URL: http://arxiv.org/abs/2405.04960v2
• Date: Mon, 17 Jun 2024 09:38:25 GMT
• Title: P-ICL: Point In-Context Learning for Named Entity Recognition with Large Language Models
• Abstract summary: In recent years, the rise of large language models (LLMs) has made it possible to directly achieve named entity recognition (NER) without any demonstration samples. Standard ICL only helps LLMs understand task instructions, format and input-label mapping, but neglects the particularity of the NER task itself. We propose a new prompting framework P-ICL to better achieve NER with LLMs, in which some point entities are leveraged as the auxiliary information to recognize each entity type.
• Score: 7.037794031385439
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: In recent years, the rise of large language models (LLMs) has made it possible to directly achieve named entity recognition (NER) without any demonstration samples or only using a few samples through in-context learning (ICL). However, standard ICL only helps LLMs understand task instructions, format and input-label mapping, but neglects the particularity of the NER task itself. In this paper, we propose a new prompting framework P-ICL to better achieve NER with LLMs, in which some point entities are leveraged as the auxiliary information to recognize each entity type. With such significant information, the LLM can achieve entity classification more precisely. To obtain optimal point entities for prompting LLMs, we also proposed a point entity selection method based on K-Means clustering. Our extensive experiments on some representative NER benchmarks verify the effectiveness of our proposed strategies in P-ICL and point entity selection.

Related papers

• Have You Ever Seen Them? Entity-level Membership Inference through Interrogating Large Language Models [6.8669133150322255]
  Large Language Models (LLMs) raise growing concerns about privacy leakage and copyright compliance.<n>We propose entity-level membership inference, which determines whether information related to a target entity is used in LLM training.<n>We construct entity-level datasets and adapt state-of-the-art sample-level label-only methods to the entity-level setting as baselines.
  arXiv  Detail & Related papers  (2026-06-22T08:40:43Z)
• Scaling Performance and Low-Resource Annotation with Many-Shot In-Context Learning for Named Entity Recognition [52.07624601400713]
  In-context learning (ICL) with large language models (LLMs) has emerged as a powerful alternative to fine-tuning for Named Entity Recognition (NER)<n>We conduct a comprehensive investigation of many-shot ICL for NER and explore its effectiveness in annotating and refining data for low-resource NER tasks.<n>Our experiments demonstrate that: (1) scaling to hundreds of in-context examples enables LLMs to match or even surpass the performance of fully supervised BERT models; and (2) using about one hundred human-labeled examples as demonstrations, many-shot in-context annotation can generate high-quality labeled
  arXiv  Detail & Related papers  (2026-06-20T05:39:59Z)
• Agentic AI for Human Resources: LLM-Driven Candidate Assessment [4.271562161453985]
  We present a modular and interpretable framework that uses Large Language Models (LLMs) to automate candidate assessment in recruitment.<n>The system integrates diverse sources, including job descriptions, CVs, interview transcripts, and HR feedback, to generate structured evaluation reports that mirror expert judgment.<n>The framework outputs detailed assessment reports, candidate comparisons, and ranked recommendations that are transparent, auditable, and suitable for real-world hiring.
  arXiv  Detail & Related papers  (2026-03-17T21:32:08Z)
• LLMs for Resource Allocation: A Participatory Budgeting Approach to Inferring Preferences [17.089038477886486]
  Large Language Models (LLMs) are increasingly expected to handle complex decision-making tasks, but their ability to perform structured resource allocation remains underexplored.<n>We present a dual-purpose framework leveraging Participatory Budgeting (PB) both as (i) a practical setting for LLM-based resource allocation and (ii) an adaptive benchmark for evaluating their reasoning capabilities.
  arXiv  Detail & Related papers  (2025-08-08T06:45:07Z)
• Label-Guided In-Context Learning for Named Entity Recognition [14.63059248497416]
  In-context learning (ICL) enables large language models to perform new tasks using only a few demonstrations.<n>We introduce DEER, a new method that leverages training labels through token-level statistics to improve ICL performance.
  arXiv  Detail & Related papers  (2025-05-29T17:54:32Z)
• IDA-Bench: Evaluating LLMs on Interactive Guided Data Analysis [60.32962597618861]
  IDA-Bench is a novel benchmark evaluating large language models in multi-round interactive scenarios.<n>Agent performance is judged by comparing its final numerical output to the human-derived baseline.<n>Even state-of-the-art coding agents (like Claude-3.7-thinking) succeed on 50% of the tasks, highlighting limitations not evident in single-turn tests.
  arXiv  Detail & Related papers  (2025-05-23T09:37:52Z)
• From Selection to Generation: A Survey of LLM-based Active Learning [153.8110509961261]
  Large Language Models (LLMs) have been employed for generating entirely new data instances and providing more cost-effective annotations. This survey aims to serve as an up-to-date resource for researchers and practitioners seeking to gain an intuitive understanding of LLM-based AL techniques.
  arXiv  Detail & Related papers  (2025-02-17T12:58:17Z)
• Aligning Large Language Models to Follow Instructions and Hallucinate Less via Effective Data Filtering [66.5524727179286]
  NOVA is a framework designed to identify high-quality data that aligns well with the learned knowledge to reduce hallucinations. It includes Internal Consistency Probing (ICP) and Semantic Equivalence Identification (SEI) to measure how familiar the LLM is with instruction data. To ensure the quality of selected samples, we introduce an expert-aligned reward model, considering characteristics beyond just familiarity.
  arXiv  Detail & Related papers  (2025-02-11T08:05:56Z)
• Unleashing the Power of Large Language Models for Group POI Recommendations [39.49785677738477]
  Group Point-of-Interest (POI) recommendations aim to predict the next POI that satisfies the diverse preferences of a group of users. Existing methods for group POI recommendations rely on single ID-based features from check-in data. We propose a framework that unleashes power of the Large Language Model (LLM) for context-aware group POI recommendations.
  arXiv  Detail & Related papers  (2024-11-20T16:02:14Z)
• Self-Calibrated Listwise Reranking with Large Language Models [137.6557607279876]
  Large language models (LLMs) have been employed in reranking tasks through a sequence-to-sequence approach. This reranking paradigm requires a sliding window strategy to iteratively handle larger candidate sets. We propose a novel self-calibrated listwise reranking method, which aims to leverage LLMs to produce global relevance scores for ranking.
  arXiv  Detail & Related papers  (2024-11-07T10:31:31Z)
• DynamicNER: A Dynamic, Multilingual, and Fine-Grained Dataset for LLM-based Named Entity Recognition [53.019885776033824]
  We propose DynamicNER, the first NER dataset designed for Large Language Models (LLMs)-based methods with dynamic categorization.<n>The dataset is also multilingual and multi-granular, covering 8 languages and 155 entity types, with corpora spanning a diverse range of domains.<n>Experiments show that DynamicNER serves as a robust and effective benchmark for LLM-based NER methods.
  arXiv  Detail & Related papers  (2024-09-17T09:32:12Z)
• Exploring Large Language Models for Feature Selection: A Data-centric Perspective [17.99621520553622]
  Large Language Models (LLMs) have influenced various domains, leveraging their exceptional few-shot and zero-shot learning capabilities. We aim to explore and understand the LLMs-based feature selection methods from a data-centric perspective. Our findings emphasize the effectiveness and robustness of text-based feature selection methods and showcase their potentials using a real-world medical application.
  arXiv  Detail & Related papers  (2024-08-21T22:35:19Z)
• Large Language Models Know What Makes Exemplary Contexts [42.90814615222177]
  In-context learning (ICL) has proven to be a significant capability with the advancement of Large Language models (LLMs) This paper presents a unified framework for LLMs that allows them to self-select influential in-context examples to compose their contexts.
  arXiv  Detail & Related papers  (2024-08-14T12:32:41Z)
• Effective Demonstration Annotation for In-Context Learning via Language Model-Based Determinantal Point Process [45.632012199451275]
  In-context learning (ICL) is a few-shot learning paradigm that involves learning mappings through input-output pairs. Existing works are highly dependent on large-scale labeled support sets, not always feasible in practical scenarios. We introduce the Language Model-based Determinant Point Process (LM-DPP) that simultaneously considers the uncertainty and diversity of unlabeled instances for optimal selection.
  arXiv  Detail & Related papers  (2024-08-04T18:08:15Z)
• Aligning Language Models with Demonstrated Feedback [58.834937450242975]
  Demonstration ITerated Task Optimization (DITTO) directly aligns language model outputs to a user's demonstrated behaviors. We evaluate DITTO's ability to learn fine-grained style and task alignment across domains such as news articles, emails, and blog posts.
  arXiv  Detail & Related papers  (2024-06-02T23:13:56Z)
• Automated Commit Message Generation with Large Language Models: An Empirical Study and Beyond [24.151927600694066]
  Commit Message Generation (CMG) approaches aim to automatically generate commit messages based on given code diffs. This paper conducts the first comprehensive experiment to investigate how far we have been in applying Large Language Models (LLMs) to generate high-quality commit messages.
  arXiv  Detail & Related papers  (2024-04-23T08:24:43Z)
• Small LLMs Are Weak Tool Learners: A Multi-LLM Agent [73.54562551341454]
  Large Language Model (LLM) agents significantly extend the capabilities of standalone LLMs. We propose a novel approach that decomposes the aforementioned capabilities into a planner, caller, and summarizer. This modular framework facilitates individual updates and the potential use of smaller LLMs for building each capability.
  arXiv  Detail & Related papers  (2024-01-14T16:17:07Z)
• Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarization [132.25202059478065]
  We benchmark large language models (LLMs) on instruction controllable text summarization. Our study reveals that instruction controllable text summarization remains a challenging task for LLMs.
  arXiv  Detail & Related papers  (2023-11-15T18:25:26Z)
• Hint-enhanced In-Context Learning wakes Large Language Models up for knowledge-intensive tasks [54.153914606302486]
  In-context learning (ICL) ability has emerged with the increasing scale of large language models (LLMs) We propose a new paradigm called Hint-enhanced In-Context Learning (HICL) to explore the power of ICL in open-domain question answering.
  arXiv  Detail & Related papers  (2023-11-03T14:39:20Z)
• LLMRec: Benchmarking Large Language Models on Recommendation Task [54.48899723591296]
  The application of Large Language Models (LLMs) in the recommendation domain has not been thoroughly investigated. We benchmark several popular off-the-shelf LLMs on five recommendation tasks, including rating prediction, sequential recommendation, direct recommendation, explanation generation, and review summarization. The benchmark results indicate that LLMs displayed only moderate proficiency in accuracy-based tasks such as sequential and direct recommendation.
  arXiv  Detail & Related papers  (2023-08-23T16:32:54Z)
• Language models are weak learners [71.33837923104808]
  We show that prompt-based large language models can operate effectively as weak learners. We incorporate these models into a boosting approach, which can leverage the knowledge within the model to outperform traditional tree-based boosting. Results illustrate the potential for prompt-based LLMs to function not just as few-shot learners themselves, but as components of larger machine learning pipelines.
  arXiv  Detail & Related papers  (2023-06-25T02:39:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.