Learning with Logical Constraints but without Shortcut Satisfaction

• URL: http://arxiv.org/abs/2403.00329v1
• Date: Fri, 1 Mar 2024 07:17:20 GMT
• Title: Learning with Logical Constraints but without Shortcut Satisfaction
• Authors: Zenan Li, Zehua Liu, Yuan Yao, Jingwei Xu, Taolue Chen, Xiaoxing Ma, Jian L\"u
• Abstract summary: We present a new framework for learning with logical constraints. Specifically, we address the shortcut satisfaction issue by introducing dual variables for logical connectives. We propose a variational framework where the encoded logical constraint is expressed as a distributional loss.
• Score: 23.219364371311084
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent studies in neuro-symbolic learning have explored the integration of logical knowledge into deep learning via encoding logical constraints as an additional loss function. However, existing approaches tend to vacuously satisfy logical constraints through shortcuts, failing to fully exploit the knowledge. In this paper, we present a new framework for learning with logical constraints. Specifically, we address the shortcut satisfaction issue by introducing dual variables for logical connectives, encoding how the constraint is satisfied. We further propose a variational framework where the encoded logical constraint is expressed as a distributional loss that is compatible with the model's original training loss. The theoretical analysis shows that the proposed approach bears salient properties, and the experimental evaluations demonstrate its superior performance in both model generalizability and constraint satisfaction.

Related papers

• Neuro-symbolic Learning Yielding Logical Constraints [22.649543443988712]
  end-to-end learning of neuro-symbolic systems is still an unsolved challenge. We propose a framework that fuses the network, symbol grounding, and logical constraint synthesisto-end learning process.
  arXiv  Detail & Related papers  (2024-10-28T12:18:25Z)
• Semantic Objective Functions: A distribution-aware method for adding logical constraints in deep learning [4.854297874710511]
  Constrained Learning and Knowledge Distillation techniques have shown promising results. We propose a loss-based method that embeds knowledge-enforces logical constraints into a machine learning model. We evaluate our method on a variety of learning tasks, including classification tasks with logic constraints.
  arXiv  Detail & Related papers  (2024-05-03T19:21:47Z)
• Logic-induced Diagnostic Reasoning for Semi-supervised Semantic Segmentation [85.12429517510311]
  LogicDiag is a neural-logic semi-supervised learning framework for semantic segmentation. Our key insight is that conflicts within pseudo labels, identified through symbolic knowledge, can serve as strong yet commonly ignored learning signals. We showcase the practical application of LogicDiag in the data-hungry segmentation scenario, where we formalize the structured abstraction of semantic concepts as a set of logic rules.
  arXiv  Detail & Related papers  (2023-08-24T06:50:07Z)
• Neuro-Symbolic Inductive Logic Programming with Logical Neural Networks [65.23508422635862]
  We propose learning rules with the recently proposed logical neural networks (LNN) Compared to others, LNNs offer strong connection to classical Boolean logic. Our experiments on standard benchmarking tasks confirm that LNN rules are highly interpretable.
  arXiv  Detail & Related papers  (2021-12-06T19:38:30Z)
• Evaluating Relaxations of Logic for Neural Networks: A Comprehensive Study [17.998891912502092]
  We study the question of how best to relax logical expressions that represent labeled examples and knowledge about a problem. We present theoretical and empirical criteria for characterizing which relaxation would perform best in various scenarios.
  arXiv  Detail & Related papers  (2021-07-28T21:16:58Z)
• Leveraging Unlabeled Data for Entity-Relation Extraction through Probabilistic Constraint Satisfaction [54.06292969184476]
  We study the problem of entity-relation extraction in the presence of symbolic domain knowledge. Our approach employs semantic loss which captures the precise meaning of a logical sentence. With a focus on low-data regimes, we show that semantic loss outperforms the baselines by a wide margin.
  arXiv  Detail & Related papers  (2021-03-20T00:16:29Z)
• Constrained Learning with Non-Convex Losses [119.8736858597118]
  Though learning has become a core technology of modern information processing, there is now ample evidence that it can lead to biased, unsafe, and prejudiced solutions.
  arXiv  Detail & Related papers  (2021-03-08T23:10:33Z)
• Reinforcement Learning with External Knowledge by using Logical Neural Networks [67.46162586940905]
  A recent neuro-symbolic framework called the Logical Neural Networks (LNNs) can simultaneously provide key-properties of both neural networks and symbolic logic. We propose an integrated method that enables model-free reinforcement learning from external knowledge sources.
  arXiv  Detail & Related papers  (2021-03-03T12:34:59Z)
• Logical Neural Networks [51.46602187496816]
  We propose a novel framework seamlessly providing key properties of both neural nets (learning) and symbolic logic (knowledge and reasoning) Every neuron has a meaning as a component of a formula in a weighted real-valued logic, yielding a highly intepretable disentangled representation. Inference is omni rather than focused on predefined target variables, and corresponds to logical reasoning.
  arXiv  Detail & Related papers  (2020-06-23T16:55:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.