IndoBERT-Relevancy: A Context-Conditioned Relevancy Classifier for Indonesian Text

• URL: http://arxiv.org/abs/2603.26095v1
• Date: Fri, 27 Mar 2026 05:58:15 GMT
• Title: IndoBERT-Relevancy: A Context-Conditioned Relevancy Classifier for Indonesian Text
• Abstract summary: We introduce IndoBERT-Relevancy, a context-conditioned relevancy classifier built on IndoBERT Large parameters.<n>We trained on a novel dataset of 31,360 labeled pairs spanning 188 topics.<n>Our final model achieves an F1 score of 0.948 and an accuracy of 96.5%, handling both formal and informal Indonesian text.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Determining whether a piece of text is relevant to a given topic is a fundamental task in natural language processing, yet it remains largely unexplored for Bahasa Indonesia. Unlike sentiment analysis or named entity recognition, relevancy classification requires the model to reason about the relationship between two inputs simultaneously: a topical context and a candidate text. We introduce IndoBERT-Relevancy, a context-conditioned relevancy classifier built on IndoBERT Large (335M parameters) and trained on a novel dataset of 31,360 labeled pairs spanning 188 topics. Through an iterative, failure-driven data construction process, we demonstrate that no single data source is sufficient for robust relevancy classification, and that targeted synthetic data can effectively address specific model weaknesses. Our final model achieves an F1 score of 0.948 and an accuracy of 96.5%, handling both formal and informal Indonesian text. The model is publicly available at HuggingFace.

Related papers

• IndoBERT-Sentiment: Context-Conditioned Sentiment Classification for Indonesian Text [0.0]
  Existing Indonesian sentiment analysis models classify text in isolation, ignoring the topical context that often determines whether a statement is positive, negative, or neutral.<n>We introduce IndoBERT-Sentiment, a context-conditioned sentiment classifier that takes both a topical context and a text as input, producing sentiment predictions grounded in the topic being discussed.<n>We show that context-conditioning, previously demonstrated for relevancy classification, transfers effectively to sentiment analysis and enables the model to correctly classify texts that are systematically misclassified by context-free approaches.
  arXiv  Detail & Related papers  (2026-04-08T13:08:33Z)
• GATE: General Arabic Text Embedding for Enhanced Semantic Textual Similarity with Matryoshka Representation Learning and Hybrid Loss Training [1.4231093967875448]
  General Arabic Text Embedding (GATE) models achieve state-of-the-art performance on the Semantic Textual Similarity task within the MTEB benchmark.<n>Gate outperforms larger models, including OpenAI, with a 20-25% performance improvement on STS benchmarks.
  arXiv  Detail & Related papers  (2025-05-30T13:29:03Z)
• Harnessing the Intrinsic Knowledge of Pretrained Language Models for Challenging Text Classification Settings [5.257719744958367]
  This thesis explores three challenging settings in text classification by leveraging the intrinsic knowledge of pretrained language models (PLMs) We develop models that utilize features based on contextualized word representations from PLMs, achieving performance that rivals or surpasses human accuracy. Lastly, we tackle the sensitivity of large language models to in-context learning prompts by selecting effective demonstrations.
  arXiv  Detail & Related papers  (2024-08-28T09:07:30Z)
• Optimal strategies to perform multilingual analysis of social content for a novel dataset in the tourism domain [5.519653885553456]
  We build the first publicly available multilingual dataset (French, English, and Spanish) for the tourism domain, composed of tourism-related tweets.<n>Our results, grounded in a novel dataset, pave the way for applying NLP to new domain-specific applications.
  arXiv  Detail & Related papers  (2023-11-20T13:08:21Z)
• Disco-Bench: A Discourse-Aware Evaluation Benchmark for Language Modelling [70.23876429382969]
  We propose a benchmark that can evaluate intra-sentence discourse properties across a diverse set of NLP tasks. Disco-Bench consists of 9 document-level testsets in the literature domain, which contain rich discourse phenomena. For linguistic analysis, we also design a diagnostic test suite that can examine whether the target models learn discourse knowledge.
  arXiv  Detail & Related papers  (2023-07-16T15:18:25Z)
• Analyzing Vietnamese Legal Questions Using Deep Neural Networks with Biaffine Classifiers [3.116035935327534]
  We propose using deep neural networks to extract important information from Vietnamese legal questions. Given a legal question in natural language, the goal is to extract all the segments that contain the needed information to answer the question.
  arXiv  Detail & Related papers  (2023-04-27T18:19:24Z)
• Distant finetuning with discourse relations for stance classification [55.131676584455306]
  We propose a new method to extract data with silver labels from raw text to finetune a model for stance classification. We also propose a 3-stage training framework where the noisy level in the data used for finetuning decreases over different stages. Our approach ranks 1st among 26 competing teams in the stance classification track of the NLPCC 2021 shared task Argumentative Text Understanding for AI Debater.
  arXiv  Detail & Related papers  (2022-04-27T04:24:35Z)
• Dynamic Hybrid Relation Network for Cross-Domain Context-Dependent Semantic Parsing [52.24507547010127]
  Cross-domain context-dependent semantic parsing is a new focus of research. We present a dynamic graph framework that effectively modelling contextual utterances, tokens, database schemas, and their complicated interaction as the conversation proceeds. The proposed framework outperforms all existing models by large margins, achieving new state-of-the-art performance on two large-scale benchmarks.
  arXiv  Detail & Related papers  (2021-01-05T18:11:29Z)
• ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning [97.10875695679499]
  We propose a novel contrastive learning framework named ERICA in pre-training phase to obtain a deeper understanding of the entities and their relations in text. Experimental results demonstrate that our proposed ERICA framework achieves consistent improvements on several document-level language understanding tasks.
  arXiv  Detail & Related papers  (2020-12-30T03:35:22Z)
• InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective [84.78604733927887]
  Large-scale language models such as BERT have achieved state-of-the-art performance across a wide range of NLP tasks. Recent studies show that such BERT-based models are vulnerable facing the threats of textual adversarial attacks. We propose InfoBERT, a novel learning framework for robust fine-tuning of pre-trained language models.
  arXiv  Detail & Related papers  (2020-10-05T20:49:26Z)
• FAT ALBERT: Finding Answers in Large Texts using Semantic Similarity Attention Layer based on BERT [0.5772546394254112]
  We develop a model based on BERT, a state-of-the-art transformer network. We are ranked first in the leader board with test accuracy of 87.79%.
  arXiv  Detail & Related papers  (2020-08-22T08:04:21Z)
• Coreferential Reasoning Learning for Language Representation [88.14248323659267]
  We present CorefBERT, a novel language representation model that can capture the coreferential relations in context. The experimental results show that, compared with existing baseline models, CorefBERT can achieve significant improvements consistently on various downstream NLP tasks.
  arXiv  Detail & Related papers  (2020-04-15T03:57:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.