Robustness of different loss functions and their impact on networks learning capability

• URL: http://arxiv.org/abs/2110.08322v1
• Date: Fri, 15 Oct 2021 19:12:42 GMT
• Title: Robustness of different loss functions and their impact on networks learning capability
• Authors: Vishal Rajput
• Abstract summary: We will look at how fast the accuracy of different models decreases when we change the pixels corresponding to the most salient gradients. We will use two sets of loss functions, generalized loss functions like Binary cross-entropy or BCE and specialized loss functions like Dice loss or focal loss.
• Score: 3.1727619150610837
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Recent developments in AI have made it ubiquitous, every industry is trying to adopt some form of intelligent processing of their data. Despite so many advances in the field, AIs full capability is yet to be exploited by the industry. Industries that involve some risk factors still remain cautious about the usage of AI due to the lack of trust in such autonomous systems. Present-day AI might be very good in a lot of things but it is very bad in reasoning and this behavior of AI can lead to catastrophic results. Autonomous cars crashing into a person or a drone getting stuck in a tree are a few examples where AI decisions lead to catastrophic results. To develop insight and generate an explanation about the learning capability of AI, we will try to analyze the working of loss functions. For our case, we will use two sets of loss functions, generalized loss functions like Binary cross-entropy or BCE and specialized loss functions like Dice loss or focal loss. Through a series of experiments, we will establish whether combining different loss functions is better than using a single loss function and if yes, then what is the reason behind it. In order to establish the difference between generalized loss and specialized losses, we will train several models using the above-mentioned losses and then compare their robustness on adversarial examples. In particular, we will look at how fast the accuracy of different models decreases when we change the pixels corresponding to the most salient gradients.

Related papers

• EvoMU: Evolutionary Machine Unlearning [13.775690509818753]
  EvoMU finds task-specific losses in the vast space of possible unlearning loss functions.<n>This work is therefore an instance of automatic scientific discovery, a.k.a. an AI co-scientist.
  arXiv  Detail & Related papers  (2026-02-02T14:19:13Z)
• The Hot Mess of AI: How Does Misalignment Scale With Model Intelligence and Task Complexity? [53.15349353876531]
  As AI becomes more capable, we entrust it with more general and consequential tasks.<n>We operationalize this question using a bias-variance decomposition of the errors made by AI models.<n>As more capable AIs pursue harder tasks, requiring more sequential action and thought, our results predict failures to be accompanied by more incoherent behavior.
  arXiv  Detail & Related papers  (2026-01-30T14:52:03Z)
• The AI off-switch problem as a signalling game: bounded rationality and incomparability [45.76759085727843]
  We model the off-switch problem as a signalling game, where a human decision-maker communicates its preferences to an AI agent. We show that a necessary condition for an AI system to refrain from disabling its off-switch is its uncertainty about the human's utility. We also analyse how message costs influence optimal strategies and extend the analysis to scenarios involving incomparability.
  arXiv  Detail & Related papers  (2025-02-10T12:44:49Z)
• What should an AI assessor optimise for? [57.96463917842822]
  An AI assessor is an external, ideally indepen-dent system that predicts an indicator, e.g., a loss value, of another AI system. Here we address the question: is it always optimal to train the assessor for the target metric? We experimentally explore this question for, respectively, regression losses and classification scores with monotonic and non-monotonic mappings.
  arXiv  Detail & Related papers  (2025-02-01T08:41:57Z)
• Raising the Stakes: Performance Pressure Improves AI-Assisted Decision Making [57.53469908423318]
  We show the effects of performance pressure on AI advice reliance when laypeople complete a common AI-assisted task. We find that when the stakes are high, people use AI advice more appropriately than when stakes are lower, regardless of the presence of an AI explanation.
  arXiv  Detail & Related papers  (2024-10-21T22:39:52Z)
• Fairness in AI and Its Long-Term Implications on Society [68.8204255655161]
  We take a closer look at AI fairness and analyze how lack of AI fairness can lead to deepening of biases over time. We discuss how biased models can lead to more negative real-world outcomes for certain groups. If the issues persist, they could be reinforced by interactions with other risks and have severe implications on society in the form of social unrest.
  arXiv  Detail & Related papers  (2023-04-16T11:22:59Z)
• The Role of Heuristics and Biases During Complex Choices with an AI Teammate [0.0]
  We argue that classic experimental methods are insufficient for studying complex choices made with AI helpers. We show that framing and anchoring effects impact how people work with an AI helper and are predictive of choice outcomes.
  arXiv  Detail & Related papers  (2023-01-14T20:06:43Z)
• AutoLossGen: Automatic Loss Function Generation for Recommender Systems [40.21831408797939]
  In recommendation systems, the choice of loss function is critical since a good loss may significantly improve the model performance. A large fraction of previous work focuses on handcrafted loss functions, which needs significant expertise and human effort. We propose an automatic loss function generation framework, AutoLossGen, which is able to generate loss functions directly constructed from basic mathematical operators.
  arXiv  Detail & Related papers  (2022-04-27T19:49:48Z)
• Do Lessons from Metric Learning Generalize to Image-Caption Retrieval? [67.45267657995748]
  The triplet loss with semi-hard negatives has become the de facto choice for image-caption retrieval (ICR) methods that are optimized from scratch. Recent progress in metric learning has given rise to new loss functions that outperform the triplet loss on tasks such as image retrieval and representation learning. We ask whether these findings generalize to the setting of ICR by comparing three loss functions on two ICR methods.
  arXiv  Detail & Related papers  (2022-02-14T15:18:00Z)
• Does Redundancy in AI Perception Systems Help to Test for Super-Human Automated Driving Performance? [6.445605125467575]
  This work reviews that it is nearly impossible to provide direct statistical evidence on the system level that this is actually the case. A commonly used strategy therefore is the use of redundancy along with the proof of sufficient subsystems' performances.
  arXiv  Detail & Related papers  (2021-12-09T08:40:31Z)
• The Who in XAI: How AI Background Shapes Perceptions of AI Explanations [61.49776160925216]
  We conduct a mixed-methods study of how two different groups--people with and without AI background--perceive different types of AI explanations. We find that (1) both groups showed unwarranted faith in numbers for different reasons and (2) each group found value in different explanations beyond their intended design.
  arXiv  Detail & Related papers  (2021-07-28T17:32:04Z)
• AI Failures: A Review of Underlying Issues [0.0]
  We focus on AI failures on account of flaws in conceptualization, design and deployment. We find that AI systems fail on account of omission and commission errors in the design of the AI system. An AI system is quite likely to fail in situations where, in effect, it is called upon to deliver moral judgments.
  arXiv  Detail & Related papers  (2020-07-18T15:31:29Z)
• Does Explainable Artificial Intelligence Improve Human Decision-Making? [17.18994675838646]
  We compare and evaluate objective human decision accuracy without AI (control), with an AI prediction (no explanation) and AI prediction with explanation. We find any kind of AI prediction tends to improve user decision accuracy, but no conclusive evidence that explainable AI has a meaningful impact. Our results indicate that, at least in some situations, the "why" information provided in explainable AI may not enhance user decision-making.
  arXiv  Detail & Related papers  (2020-06-19T15:46:13Z)
• Is the Most Accurate AI the Best Teammate? Optimizing AI for Teamwork [54.309495231017344]
  We argue that AI systems should be trained in a human-centered manner, directly optimized for team performance. We study this proposal for a specific type of human-AI teaming, where the human overseer chooses to either accept the AI recommendation or solve the task themselves. Our experiments with linear and non-linear models on real-world, high-stakes datasets show that the most accuracy AI may not lead to highest team performance.
  arXiv  Detail & Related papers  (2020-04-27T19:06:28Z)
• On Adversarial Examples and Stealth Attacks in Artificial Intelligence Systems [62.997667081978825]
  We present a formal framework for assessing and analyzing two classes of malevolent action towards generic Artificial Intelligence (AI) systems. The first class involves adversarial examples and concerns the introduction of small perturbations of the input data that cause misclassification. The second class, introduced here for the first time and named stealth attacks, involves small perturbations to the AI system itself.
  arXiv  Detail & Related papers  (2020-04-09T10:56:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.