Finding Input Characterizations for Output Properties in ReLU Neural Networks

• URL: http://arxiv.org/abs/2003.04273v1
• Date: Mon, 9 Mar 2020 17:29:39 GMT
• Title: Finding Input Characterizations for Output Properties in ReLU Neural Networks
• Abstract summary: Deep Neural Networks (DNNs) have emerged as a powerful mechanism and are being increasingly deployed in real-world safety-critical domains. Despite the widespread success, their complex architecture makes proving any formal guarantees about them difficult. We extend the ideas presented in and introduce a way to bridge the gap between the architecture and the high-level specifications.
• Score: 0.9614694312155797
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep Neural Networks (DNNs) have emerged as a powerful mechanism and are being increasingly deployed in real-world safety-critical domains. Despite the widespread success, their complex architecture makes proving any formal guarantees about them difficult. Identifying how logical notions of high-level correctness relate to the complex low-level network architecture is a significant challenge. In this project, we extend the ideas presented in and introduce a way to bridge the gap between the architecture and the high-level specifications. Our key insight is that instead of directly proving the safety properties that are required, we first prove properties that relate closely to the structure of the neural net and use them to reason about the safety properties. We build theoretical foundations for our approach, and empirically evaluate the performance through various experiments, achieving promising results than the existing approach by identifying a larger region of input space that guarantees a certain property on the output.

Related papers

• veriFIRE: an Industrial Case Study in Verifying Consistency Properties for a DNN-Based Wildfire Detection System [4.688090702715578]
  We present an end-to-end methodology for verifying textitconsistency properties in a system.<n>Our approach encodes application-grounded requirements into solver-compatible queries for existing neural network verifiers.<n>Results demonstrate that meaningful, domain-specific guarantees can be obtained for industrial systems.
  arXiv  Detail & Related papers  (2026-06-02T18:32:03Z)
• CLAIRE: Compressed Latent Autoencoder for Industrial Representation and Evaluation -- A Deep Learning Framework for Smart Manufacturing [51.56484100374058]
  CLAIRE is a hybrid end-to-end learning framework that integrates unsupervised deep representation learning with supervised classification for intelligent quality control in smart manufacturing systems.<n>It employs an optimized deep autoencoder to transform raw input into a compact latent space, effectively capturing the intrinsic data structure while suppressing irrelevant or noisy features.<n>The proposed framework highlights the potential of integrating explainable AI with feature-aware regularization for robust fault detection.
  arXiv  Detail & Related papers  (2026-03-06T15:11:58Z)
• Distributed Security: From Isolated Properties to Synergistic Trust [5.240957672918797]
  This vision paper argues for a fundamental shift in how we approach distributed security.<n>We conclude four foundational properties, textitagreement, consistency, privacy, verifiability, accountability<n>We identify critical research challenges: discovering new security properties driven by emerging applications, managing the computational overhead of cryptographic primitives in high-performance consensus layers, and addressing post-quantum and human-factor challenges.
  arXiv  Detail & Related papers  (2026-02-20T08:31:18Z)
• CogRail: Benchmarking VLMs in Cognitive Intrusion Perception for Intelligent Railway Transportation Systems [29.385460126069386]
  We introduce a novel benchmark, CogRail, which integrates curated datasets with cognitively driven question-answer annotations.<n>Building upon this benchmark, we conduct a systematic evaluation of state-of-the-art visual-language models.<n>We propose a joint fine-tuning framework that integrates three core tasks, position perception, movement prediction, and threat analysis.
  arXiv  Detail & Related papers  (2026-01-14T16:36:26Z)
• Closed-Loop LLM Discovery of Non-Standard Channel Priors in Vision Models [48.83701310501069]
  Large Language Models (LLMs) offer a transformative approach to Neural Architecture Search (NAS)<n>We formulate the search as a sequence of conditional code generation tasks, where an LLM refines architectural specifications based on performance telemetry.<n>We generate a vast corpus of valid, shape-consistent architectures via Abstract Syntax Tree (AST) mutations.<n> Experimental results on CIFAR-100 validate the efficacy of this approach, demonstrating that the model yields statistically significant improvements in accuracy.
  arXiv  Detail & Related papers  (2026-01-13T13:00:30Z)
• Identifiability of Deep Polynomial Neural Networks [2.747263800047993]
  Polynomial Neural Networks (PNNs) possess a rich algebraic and geometric structure.<n>Their identifiability -- a key property for ensuring interpretability -- remains poorly understood.<n>We present a comprehensive analysis of the identifiability of deep PNNs.
  arXiv  Detail & Related papers  (2025-06-20T15:58:46Z)
• Hierarchical Uncertainty-Aware Graph Neural Network [1.7495213911983414]
  We introduce the Hierarchical Uncertainty-Aware Graph Neural Network (HU-GNN), which unifies multi-scale representation learning, principled uncertainty estimation, and self-supervised embedding diversity within a single end-to-end framework. Specifically, HU-GNN adaptively forms node clusters and estimates uncertainty at multiple structural scales from individual nodes to higher levels.
  arXiv  Detail & Related papers  (2025-04-28T14:22:18Z)
• Hadamard product in deep learning: Introduction, Advances and Challenges [68.26011575333268]
  This survey examines a fundamental yet understudied primitive: the Hadamard product. Despite its widespread implementation across various applications, the Hadamard product has not been systematically analyzed as a core architectural primitive. We present the first comprehensive taxonomy of its applications in deep learning, identifying four principal domains: higher-order correlation, multimodal data fusion, dynamic representation modulation, and efficient pairwise operations.
  arXiv  Detail & Related papers  (2025-04-17T17:26:29Z)
• Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domains [92.36624674516553]
  Reinforcement learning with verifiable rewards (RLVR) has demonstrated significant success in enhancing mathematical reasoning and coding performance of large language models (LLMs) We investigate the effectiveness and scalability of RLVR across diverse real-world domains including medicine, chemistry, psychology, economics, and education. We utilize a generative scoring technique that yields soft, model-based reward signals to overcome limitations posed by binary verifications.
  arXiv  Detail & Related papers  (2025-03-31T08:22:49Z)
• StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization [94.31508613367296]
  Retrieval-augmented generation (RAG) is a key means to effectively enhance large language models (LLMs) We propose StructRAG, which can identify the optimal structure type for the task at hand, reconstruct original documents into this structured format, and infer answers based on the resulting structure. Experiments show that StructRAG achieves state-of-the-art performance, particularly excelling in challenging scenarios.
  arXiv  Detail & Related papers  (2024-10-11T13:52:44Z)
• Robust NAS under adversarial training: benchmark, theory, and beyond [55.51199265630444]
  We release a comprehensive data set that encompasses both clean accuracy and robust accuracy for a vast array of adversarially trained networks. We also establish a generalization theory for searching architecture in terms of clean accuracy and robust accuracy under multi-objective adversarial training.
  arXiv  Detail & Related papers  (2024-03-19T20:10:23Z)
• Towards Improving Robustness Against Common Corruptions using Mixture of Class Specific Experts [10.27974860479791]
  This paper introduces a novel paradigm known as the Mixture of Class-Specific Expert Architecture. The proposed architecture aims to mitigate vulnerabilities associated with common neural network structures.
  arXiv  Detail & Related papers  (2023-11-16T20:09:47Z)
• Enumerating Safe Regions in Deep Neural Networks with Provable Probabilistic Guarantees [86.1362094580439]
  We introduce the AllDNN-Verification problem: given a safety property and a DNN, enumerate the set of all the regions of the property input domain which are safe. Due to the #P-hardness of the problem, we propose an efficient approximation method called epsilon-ProVe. Our approach exploits a controllable underestimation of the output reachable sets obtained via statistical prediction of tolerance limits.
  arXiv  Detail & Related papers  (2023-08-18T22:30:35Z)
• Neural Architecture Design and Robustness: A Dataset [11.83842808044211]
  We introduce a database on neural architecture design and robustness evaluations. We evaluate all these networks on a range of common adversarial attacks and corruption types. We find that carefully crafting the topology of a network can have substantial impact on its robustness.
  arXiv  Detail & Related papers  (2023-06-11T16:02:14Z)
• Understanding Robust Learning through the Lens of Representation Similarities [37.66877172364004]
  robustness to adversarial examples has emerged as a desirable property for deep neural networks (DNNs) In this paper, we aim to understand how the properties of representations learned by robust training differ from those obtained from standard, non-robust training.
  arXiv  Detail & Related papers  (2022-06-20T16:06:20Z)
• Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks [98.21130211336964]
  Deep neural networks (DNNs) are known to be vulnerable to adversarial attacks. In this paper, we investigate the impact of network width and depth on the robustness of adversarially trained DNNs.
  arXiv  Detail & Related papers  (2021-10-07T23:13:33Z)
• Learning Structures for Deep Neural Networks [99.8331363309895]
  We propose to adopt the efficient coding principle, rooted in information theory and developed in computational neuroscience. We show that sparse coding can effectively maximize the entropy of the output signals. Our experiments on a public image classification dataset demonstrate that using the structure learned from scratch by our proposed algorithm, one can achieve a classification accuracy comparable to the best expert-designed structure.
  arXiv  Detail & Related papers  (2021-05-27T12:27:24Z)
• Deep Feature Space: A Geometrical Perspective [2.763016063783766]
  We present a novel way of visualizing and understanding the vector space before the NNs' output layer. Main attention is paid to the nature of overfitting in the feature space and its adverse effect on further exploitation.
  arXiv  Detail & Related papers  (2020-06-30T18:51:02Z)
• Understanding Deep Architectures with Reasoning Layer [60.90906477693774]
  We show that properties of the algorithm layers, such as convergence, stability, and sensitivity, are intimately related to the approximation and generalization abilities of the end-to-end model. Our theory can provide useful guidelines for designing deep architectures with reasoning layers.
  arXiv  Detail & Related papers  (2020-06-24T00:26:35Z)
• Global Robustness Verification Networks [33.52550848953545]
  We develop a global robustness verification framework with three components. New network architecture Sliding Door Network (SDN) enabling feasible rule-based back-propagation'' We demonstrate the effectiveness of our approach on both synthetic and real datasets.
  arXiv  Detail & Related papers  (2020-06-08T08:09:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.