PyEuroVoc: A Tool for Multilingual Legal Document Classification with EuroVoc Descriptors

• URL: http://arxiv.org/abs/2108.01139v2
• Date: Wed, 4 Aug 2021 18:55:38 GMT
• Title: PyEuroVoc: A Tool for Multilingual Legal Document Classification with EuroVoc Descriptors
• Authors: Andrei-Marius Avram, Vasile Pais, Dan Tufis
• Abstract summary: We propose a unified framework for EuroVoc classification on 22 languages by fine-tuning modern Transformer-based pretrained language models. The code and the fine-tuned models were open sourced, together with a programmatic interface that eases the process of loading the weights of a trained model and of classifying a new document.
• Score: 0.3007949058551534
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: EuroVoc is a multilingual thesaurus that was built for organizing the legislative documentary of the European Union institutions. It contains thousands of categories at different levels of specificity and its descriptors are targeted by legal texts in almost thirty languages. In this work we propose a unified framework for EuroVoc classification on 22 languages by fine-tuning modern Transformer-based pretrained language models. We study extensively the performance of our trained models and show that they significantly improve the results obtained by a similar tool - JEX - on the same dataset. The code and the fine-tuned models were open sourced, together with a programmatic interface that eases the process of loading the weights of a trained model and of classifying a new document.

Related papers

• CroissantLLM: A Truly Bilingual French-English Language Model [42.03897426049679]
  We introduce CroissantLLM, a 1.3B language model pretrained on a set of 3T English and French tokens. We pioneer the approach of training an intrinsically bilingual model with a 1:1 English-to-French pretraining data ratio. To assess performance outside of English, we craft a novel benchmark, FrenchBench.
  arXiv  Detail & Related papers  (2024-02-01T17:17:55Z)
• Soft Language Clustering for Multilingual Model Pre-training [57.18058739931463]
  We propose XLM-P, which contextually retrieves prompts as flexible guidance for encoding instances conditionally. Our XLM-P enables (1) lightweight modeling of language-invariant and language-specific knowledge across languages, and (2) easy integration with other multilingual pre-training methods.
  arXiv  Detail & Related papers  (2023-06-13T08:08:08Z)
• T3L: Translate-and-Test Transfer Learning for Cross-Lingual Text Classification [50.675552118811]
  Cross-lingual text classification is typically built on large-scale, multilingual language models (LMs) pretrained on a variety of languages of interest. We propose revisiting the classic "translate-and-test" pipeline to neatly separate the translation and classification stages.
  arXiv  Detail & Related papers  (2023-06-08T07:33:22Z)
• Multi-lingual Evaluation of Code Generation Models [82.7357812992118]
  We present new benchmarks on evaluation code generation models: MBXP and Multilingual HumanEval, and MathQA-X. These datasets cover over 10 programming languages. We are able to assess the performance of code generation models in a multi-lingual fashion.
  arXiv  Detail & Related papers  (2022-10-26T17:17:06Z)
• mGPT: Few-Shot Learners Go Multilingual [1.4354798873010843]
  This paper introduces two autoregressive GPT-like models with 1.3 billion and 13 billion parameters trained on 60 languages. We reproduce the GPT-3 architecture using GPT-2 sources and the sparse attention mechanism. The resulting models show performance on par with the recently released XGLM models by Facebook.
  arXiv  Detail & Related papers  (2022-04-15T13:02:33Z)
• Models and Datasets for Cross-Lingual Summarisation [78.56238251185214]
  We present a cross-lingual summarisation corpus with long documents in a source language associated with multi-sentence summaries in a target language. The corpus covers twelve language pairs and directions for four European languages, namely Czech, English, French and German. We derive cross-lingual document-summary instances from Wikipedia by combining lead paragraphs and articles' bodies from language aligned Wikipedia titles.
  arXiv  Detail & Related papers  (2022-02-19T11:55:40Z)
• A Multilingual Bag-of-Entities Model for Zero-Shot Cross-Lingual Text Classification [16.684856745734944]
  We present a multilingual bag-of-entities model that boosts the performance of zero-shot cross-lingual text classification. It leverages the multilingual nature of Wikidata: entities in multiple languages representing the same concept are defined with a unique identifier. A model trained on entity features in a resource-rich language can thus be directly applied to other languages.
  arXiv  Detail & Related papers  (2021-10-15T01:10:50Z)
• Exploring Teacher-Student Learning Approach for Multi-lingual Speech-to-Intent Classification [73.5497360800395]
  We develop an end-to-end system that supports multiple languages. We exploit knowledge from a pre-trained multi-lingual natural language processing model.
  arXiv  Detail & Related papers  (2021-09-28T04:43:11Z)
• MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer [13.24356999779404]
  We introduce MULTI-EURLEX, a new multilingual dataset for topic classification of legal documents. The dataset comprises 65k European Union (EU) laws, officially translated in 23 languages, annotated with multiple labels from the EUROVOC taxonomy. We use the dataset as a testbed for zero-shot cross-lingual transfer, where we exploit annotated training documents in one language (source) to classify documents in another language (target)
  arXiv  Detail & Related papers  (2021-09-02T12:52:55Z)
• Scalable Cross-lingual Document Similarity through Language-specific Concept Hierarchies [0.0]
  This paper presents an unsupervised document similarity algorithm that does not require parallel or comparable corpora. The algorithm annotates topics automatically created from documents in a single language with cross-lingual labels. Experiments performed on the English, Spanish and French editions of JCR-Acquis corpora reveal promising results on classifying and sorting documents by similar content.
  arXiv  Detail & Related papers  (2020-12-15T10:42:40Z)
• Unsupervised Domain Adaptation of a Pretrained Cross-Lingual Language Model [58.27176041092891]
  Recent research indicates that pretraining cross-lingual language models on large-scale unlabeled texts yields significant performance improvements. We propose a novel unsupervised feature decomposition method that can automatically extract domain-specific features from the entangled pretrained cross-lingual representations. Our proposed model leverages mutual information estimation to decompose the representations computed by a cross-lingual model into domain-invariant and domain-specific parts.
  arXiv  Detail & Related papers  (2020-11-23T16:00:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.