Semantic Objective Functions: A distribution-aware method for adding logical constraints in deep learning

• URL: http://arxiv.org/abs/2405.15789v1
• Date: Fri, 3 May 2024 19:21:47 GMT
• Title: Semantic Objective Functions: A distribution-aware method for adding logical constraints in deep learning
• Authors: Miguel Angel Mendez-Lucero, Enrique Bojorquez Gallardo, Vaishak Belle,
• Abstract summary: Constrained Learning and Knowledge Distillation techniques have shown promising results. We propose a loss-based method that embeds knowledge-enforces logical constraints into a machine learning model. We evaluate our method on a variety of learning tasks, including classification tasks with logic constraints.
• Score: 4.854297874710511
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Issues of safety, explainability, and efficiency are of increasing concern in learning systems deployed with hard and soft constraints. Symbolic Constrained Learning and Knowledge Distillation techniques have shown promising results in this area, by embedding and extracting knowledge, as well as providing logical constraints during neural network training. Although many frameworks exist to date, through an integration of logic and information geometry, we provide a construction and theoretical framework for these tasks that generalize many approaches. We propose a loss-based method that embeds knowledge-enforces logical constraints-into a machine learning model that outputs probability distributions. This is done by constructing a distribution from the external knowledge/logic formula and constructing a loss function as a linear combination of the original loss function with the Fisher-Rao distance or Kullback-Leibler divergence to the constraint distribution. This construction includes logical constraints in the form of propositional formulas (Boolean variables), formulas of a first-order language with finite variables over a model with compact domain (categorical and continuous variables), and in general, likely applicable to any statistical model that was pretrained with semantic information. We evaluate our method on a variety of learning tasks, including classification tasks with logic constraints, transferring knowledge from logic formulas, and knowledge distillation from general distributions.

Related papers

• Neuro-symbolic Learning Yielding Logical Constraints [22.649543443988712]
  end-to-end learning of neuro-symbolic systems is still an unsolved challenge. We propose a framework that fuses the network, symbol grounding, and logical constraint synthesisto-end learning process.
  arXiv  Detail & Related papers  (2024-10-28T12:18:25Z)
• Learning with Logical Constraints but without Shortcut Satisfaction [23.219364371311084]
  We present a new framework for learning with logical constraints. Specifically, we address the shortcut satisfaction issue by introducing dual variables for logical connectives. We propose a variational framework where the encoded logical constraint is expressed as a distributional loss.
  arXiv  Detail & Related papers  (2024-03-01T07:17:20Z)
• Logic-induced Diagnostic Reasoning for Semi-supervised Semantic Segmentation [85.12429517510311]
  LogicDiag is a neural-logic semi-supervised learning framework for semantic segmentation. Our key insight is that conflicts within pseudo labels, identified through symbolic knowledge, can serve as strong yet commonly ignored learning signals. We showcase the practical application of LogicDiag in the data-hungry segmentation scenario, where we formalize the structured abstraction of semantic concepts as a set of logic rules.
  arXiv  Detail & Related papers  (2023-08-24T06:50:07Z)
• PROTOtypical Logic Tensor Networks (PROTO-LTN) for Zero Shot Learning [2.236663830879273]
  Logic Networks (LTNs) are neuro-symbolic systems based on a differentiable, first-order logic grounded into a deep neural network. We focus here on the subsumption or textttisOfClass predicate, which is fundamental to encode most semantic image interpretation tasks. We propose a common textttisOfClass predicate, whose level of truth is a function of the distance between an object embedding and the corresponding class prototype.
  arXiv  Detail & Related papers  (2022-06-26T18:34:07Z)
• Logical blocks for fault-tolerant topological quantum computation [55.41644538483948]
  We present a framework for universal fault-tolerant logic motivated by the need for platform-independent logical gate definitions. We explore novel schemes for universal logic that improve resource overheads. Motivated by the favorable logical error rates for boundaryless computation, we introduce a novel computational scheme.
  arXiv  Detail & Related papers  (2021-12-22T19:00:03Z)
• Leveraging Unlabeled Data for Entity-Relation Extraction through Probabilistic Constraint Satisfaction [54.06292969184476]
  We study the problem of entity-relation extraction in the presence of symbolic domain knowledge. Our approach employs semantic loss which captures the precise meaning of a logical sentence. With a focus on low-data regimes, we show that semantic loss outperforms the baselines by a wide margin.
  arXiv  Detail & Related papers  (2021-03-20T00:16:29Z)
• Reinforcement Learning with External Knowledge by using Logical Neural Networks [67.46162586940905]
  A recent neuro-symbolic framework called the Logical Neural Networks (LNNs) can simultaneously provide key-properties of both neural networks and symbolic logic. We propose an integrated method that enables model-free reinforcement learning from external knowledge sources.
  arXiv  Detail & Related papers  (2021-03-03T12:34:59Z)
• Logical Neural Networks [51.46602187496816]
  We propose a novel framework seamlessly providing key properties of both neural nets (learning) and symbolic logic (knowledge and reasoning) Every neuron has a meaning as a component of a formula in a weighted real-valued logic, yielding a highly intepretable disentangled representation. Inference is omni rather than focused on predefined target variables, and corresponds to logical reasoning.
  arXiv  Detail & Related papers  (2020-06-23T16:55:45Z)
• An Integer Linear Programming Framework for Mining Constraints from Data [81.60135973848125]
  We present a general framework for mining constraints from data. In particular, we consider the inference in structured output prediction as an integer linear programming (ILP) problem. We show that our approach can learn to solve 9x9 Sudoku puzzles and minimal spanning tree problems from examples without providing the underlying rules.
  arXiv  Detail & Related papers  (2020-06-18T20:09:53Z)
• Relational Neural Machines [19.569025323453257]
  This paper presents a novel framework allowing jointly train the parameters of the learners and of a First-Order Logic based reasoner. A Neural Machine is able recover both classical learning results in case of pure sub-symbolic learning, and Markov Logic Networks. Proper algorithmic solutions are devised to make learning and inference tractable in large-scale problems.
  arXiv  Detail & Related papers  (2020-02-06T10:53:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.