Solving morphological analogies: from retrieval to generation

• URL: http://arxiv.org/abs/2303.18062v2
• Date: Wed, 17 Apr 2024 15:23:12 GMT
• Title: Solving morphological analogies: from retrieval to generation
• Authors: Esteban Marquer, Miguel Couceiro,
• Abstract summary: Analogical inference is a capability of human reasoning, and has been used to solve hard reasoning tasks. We propose a deep learning (DL) framework to address and tackle two key tasks in AR: analogy detection and solving. The framework is thoroughly tested on the Siganalogies dataset of morphological analogical proportions (APs) between words, and shown to outperform symbolic approaches in many languages.
• Score: 4.834203844100681
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Analogical inference is a remarkable capability of human reasoning, and has been used to solve hard reasoning tasks. Analogy based reasoning (AR) has gained increasing interest from the artificial intelligence community and has shown its potential in multiple machine learning tasks such as classification, decision making and recommendation with competitive results. We propose a deep learning (DL) framework to address and tackle two key tasks in AR: analogy detection and solving. The framework is thoroughly tested on the Siganalogies dataset of morphological analogical proportions (APs) between words, and shown to outperform symbolic approaches in many languages. Previous work have explored the behavior of the Analogy Neural Network for classification (ANNc) on analogy detection and of the Analogy Neural Network for retrieval (ANNr) on analogy solving by retrieval, as well as the potential of an autoencoder (AE) for analogy solving by generating the solution word. In this article we summarize these findings and we extend them by combining ANNr and the AE embedding model, and checking the performance of ANNc as an retrieval method. The combination of ANNr and AE outperforms the other approaches in almost all cases, and ANNc as a retrieval method achieves competitive or better performance than 3CosMul. We conclude with general guidelines on using our framework to tackle APs with DL.

Related papers

• Binary Code Similarity Detection via Graph Contrastive Learning on Intermediate Representations [52.34030226129628]
  Binary Code Similarity Detection (BCSD) plays a crucial role in numerous fields, including vulnerability detection, malware analysis, and code reuse identification. In this paper, we propose IRBinDiff, which mitigates compilation differences by leveraging LLVM-IR with higher-level semantic abstraction. Our extensive experiments, conducted under varied compilation settings, demonstrate that IRBinDiff outperforms other leading BCSD methods in both One-to-one comparison and One-to-many search scenarios.
  arXiv  Detail & Related papers  (2024-10-24T09:09:20Z)
• ARN: Analogical Reasoning on Narratives [13.707344123755126]
  We develop a framework that operationalizes dominant theories of analogy, using narrative elements to create surface and system mappings. We show that while all LLMs can largely recognize near analogies, even the largest ones struggle with far analogies in a zero-shot setting.
  arXiv  Detail & Related papers  (2023-10-02T08:58:29Z)
• RAVEN: In-Context Learning with Retrieval-Augmented Encoder-Decoder Language Models [57.12888828853409]
  RAVEN is a model that combines retrieval-augmented masked language modeling and prefix language modeling. Fusion-in-Context Learning enables the model to leverage more in-context examples without requiring additional training. Our work underscores the potential of retrieval-augmented encoder-decoder language models for in-context learning.
  arXiv  Detail & Related papers  (2023-08-15T17:59:18Z)
• ANALOGYKB: Unlocking Analogical Reasoning of Language Models with A Million-scale Knowledge Base [51.777618249271725]
  ANALOGYKB is a million-scale analogy knowledge base derived from existing knowledge graphs (KGs) It identifies two types of analogies from the KGs: 1) analogies of the same relations, which can be directly extracted from the KGs, and 2) analogies of analogous relations, which are identified with a selection and filtering pipeline enabled by large language models (LLMs)
  arXiv  Detail & Related papers  (2023-05-10T09:03:01Z)
• Nested Named Entity Recognition from Medical Texts: An Adaptive Shared Network Architecture with Attentive CRF [53.55504611255664]
  We propose a novel method, referred to as ASAC, to solve the dilemma caused by the nested phenomenon. The proposed method contains two key modules: the adaptive shared (AS) part and the attentive conditional random field (ACRF) module. Our model could learn better entity representations by capturing the implicit distinctions and relationships between different categories of entities.
  arXiv  Detail & Related papers  (2022-11-09T09:23:56Z)
• LogiGAN: Learning Logical Reasoning via Adversarial Pre-training [58.11043285534766]
  We present LogiGAN, an unsupervised adversarial pre-training framework for improving logical reasoning abilities of language models. Inspired by the facilitation effect of reflective thinking in human learning, we simulate the learning-thinking process with an adversarial Generator-Verifier architecture. Both base and large size language models pre-trained with LogiGAN demonstrate obvious performance improvement on 12 datasets.
  arXiv  Detail & Related papers  (2022-05-18T08:46:49Z)
• Deep Learning Reproducibility and Explainable AI (XAI) [9.13755431537592]
  The nondeterminism of Deep Learning (DL) training algorithms and its influence on the explainability of neural network (NN) models are investigated. To discuss the issue, two convolutional neural networks (CNN) have been trained and their results compared.
  arXiv  Detail & Related papers  (2022-02-23T12:06:20Z)
• Tackling Morphological Analogies Using Deep Learning -- Extended Version [8.288496996031684]
  Analogical proportions are statements of the form "A is to B as C is to D" We propose an approach using deep learning to detect and solve morphological analogies. We demonstrate our model's competitive performance on analogy detection and resolution over multiple languages.
  arXiv  Detail & Related papers  (2021-11-09T13:45:23Z)
• A Neural Approach for Detecting Morphological Analogies [7.89271130004391]
  Analogical proportions are statements of the form "A is to B as C is to D" We propose a deep learning approach to detect morphological analogies.
  arXiv  Detail & Related papers  (2021-08-09T11:21:55Z)
• Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
  There is an increasing need for active learning algorithms that are compatible with deep neural networks. This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
  arXiv  Detail & Related papers  (2021-06-17T17:26:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.