Automatic and Efficient Customization of Neural Networks for ML Applications

• URL: http://arxiv.org/abs/2310.04685v1
• Date: Sat, 7 Oct 2023 04:13:29 GMT
• Title: Automatic and Efficient Customization of Neural Networks for ML Applications
• Authors: Yuhan Liu, Chengcheng Wan, Kuntai Du, Henry Hoffmann, Junchen Jiang, Shan Lu, Michael Maire
• Abstract summary: We propose ChameleonAPI, which takes effect without changing the application source code. ChameleonAPI uses the loss function to efficiently train a neural network model customized for each application. Compared to a baseline that selects the best-of-all commercial ML API, we show that ChameleonAPI reduces incorrect application decisions by 43%.
• Score: 29.391143085794184
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: ML APIs have greatly relieved application developers of the burden to design and train their own neural network models -- classifying objects in an image can now be as simple as one line of Python code to call an API. However, these APIs offer the same pre-trained models regardless of how their output is used by different applications. This can be suboptimal as not all ML inference errors can cause application failures, and the distinction between inference errors that can or cannot cause failures varies greatly across applications. To tackle this problem, we first study 77 real-world applications, which collectively use six ML APIs from two providers, to reveal common patterns of how ML API output affects applications' decision processes. Inspired by the findings, we propose ChameleonAPI, an optimization framework for ML APIs, which takes effect without changing the application source code. ChameleonAPI provides application developers with a parser that automatically analyzes the application to produce an abstract of its decision process, which is then used to devise an application-specific loss function that only penalizes API output errors critical to the application. ChameleonAPI uses the loss function to efficiently train a neural network model customized for each application and deploys it to serve API invocations from the respective application via existing interface. Compared to a baseline that selects the best-of-all commercial ML API, we show that ChameleonAPI reduces incorrect application decisions by 43%.

Related papers

• LLM-assisted Mutation for Whitebox API Testing [40.91007243855959]
  MioHint is a novel white-box API testing approach that leverages the code comprehension capabilities of Large Language Model (LLM) to boost API testing. To evaluate the effectiveness of our method, we conducted experiments across 16 real-world API services.
  arXiv  Detail & Related papers  (2025-04-08T07:14:51Z)
• Identifying and Mitigating API Misuse in Large Language Models [26.4403427473915]
  API misuse in code generated by large language models (LLMs) represents a serious emerging challenge in software development. This paper presents the first comprehensive study of API misuse patterns in LLM-generated code, analyzing both method selection and parameter usage across Python and Java. We propose Dr.Fix, a novel LLM-based automatic program repair approach for API misuse based on the aforementioned taxonomy.
  arXiv  Detail & Related papers  (2025-03-28T18:43:12Z)
• Reinforcement Learning for Long-Horizon Interactive LLM Agents [56.9860859585028]
  Interactive digital agents (IDAs) leverage APIs of stateful digital environments to perform tasks in response to user requests. We present a reinforcement learning (RL) approach that trains IDAs directly in their target environments. We derive LOOP, a data- and memory-efficient variant of proximal policy optimization.
  arXiv  Detail & Related papers  (2025-02-03T18:35:42Z)
• Your Fix Is My Exploit: Enabling Comprehensive DL Library API Fuzzing with Large Language Models [49.214291813478695]
  Deep learning (DL) libraries, widely used in AI applications, often contain vulnerabilities like overflows and use buffer-free errors. Traditional fuzzing struggles with the complexity and API diversity of DL libraries. We propose DFUZZ, an LLM-driven fuzzing approach for DL libraries.
  arXiv  Detail & Related papers  (2025-01-08T07:07:22Z)
• ExploraCoder: Advancing code generation for multiple unseen APIs via planning and chained exploration [70.26807758443675]
  ExploraCoder is a training-free framework that empowers large language models to invoke unseen APIs in code solution. We show that ExploraCoder significantly improves performance for models lacking prior API knowledge, achieving an absolute increase of 11.24% over niave RAG approaches and 14.07% over pretraining methods in pass@10.
  arXiv  Detail & Related papers  (2024-12-06T19:00:15Z)
• Model Equality Testing: Which Model Is This API Serving? [59.005869726179455]
  We formalize detecting such distortions as Model Equality Testing, a two-sample testing problem. A test built on a simple string kernel achieves a median of 77.4% power against a range of distortions. We then apply this test to commercial inference APIs for four Llama models, finding that 11 out of 31 endpoints serve different distributions than reference weights released by Meta.
  arXiv  Detail & Related papers  (2024-10-26T18:34:53Z)
• AppBench: Planning of Multiple APIs from Various APPs for Complex User Instruction [24.67142048995415]
  Large Language Models (LLMs) can interact with the real world by connecting with versatile external APIs. We introduce textttAppBench, the first benchmark to evaluate LLMs' ability to plan and execute multiple APIs from various sources.
  arXiv  Detail & Related papers  (2024-10-10T04:03:13Z)
• WorldAPIs: The World Is Worth How Many APIs? A Thought Experiment [49.00213183302225]
  We propose a framework to induce new APIs by grounding wikiHow instruction to situated agent policies. Inspired by recent successes in large language models (LLMs) for embodied planning, we propose a few-shot prompting to steer GPT-4.
  arXiv  Detail & Related papers  (2024-07-10T15:52:44Z)
• A Solution-based LLM API-using Methodology for Academic Information Seeking [49.096714812902576]
  SoAy is a solution-based LLM API-using methodology for academic information seeking. It uses code with a solution as the reasoning method, where a solution is a pre-constructed API calling sequence. Results show a 34.58-75.99% performance improvement compared to state-of-the-art LLM API-based baselines.
  arXiv  Detail & Related papers  (2024-05-24T02:44:14Z)
• Octopus: On-device language model for function calling of software APIs [9.78611123915888]
  Large Language Models (LLMs) play a crucial role due to their advanced text processing and generation abilities. This study introduces a new strategy aimed at harnessing on-device LLMs in invoking software APIs.
  arXiv  Detail & Related papers  (2024-04-02T01:29:28Z)
• Continual Learning From a Stream of APIs [90.41825351073908]
  Continual learning (CL) aims to learn new tasks without forgetting previous tasks. Existing CL methods require a large amount of raw data, which is often unavailable due to copyright considerations and privacy risks. This paper considers two practical-yet-novel CL settings: data-efficient CL (DECL-APIs) and data-free CL (DFCL-APIs)
  arXiv  Detail & Related papers  (2023-08-31T11:16:00Z)
• Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models [187.58051653991686]
  Large language models (LLMs) have achieved remarkable progress in solving various natural language processing tasks. However, they have inherent limitations as they are incapable of accessing up-to-date information. We present Chameleon, an AI system that augments LLMs with plug-and-play modules for compositional reasoning.
  arXiv  Detail & Related papers  (2023-04-19T17:47:47Z)
• HAPI: A Large-scale Longitudinal Dataset of Commercial ML API Predictions [35.48276161473216]
  We present HAPI, a longitudinal dataset of 1,761,417 instances of commercial ML API applications. Each instance consists of a query input for an API along with the API's output prediction/annotation and confidence scores.
  arXiv  Detail & Related papers  (2022-09-18T01:52:16Z)
• Petals: Collaborative Inference and Fine-tuning of Large Models [78.37798144357977]
  Many NLP tasks benefit from using large language models (LLMs) that often have more than 100 billion parameters. With the release of BLOOM-176B and OPT-175B, everyone can download pretrained models of this scale. We propose Petals $-$ a system for inference and fine-tuning of large models collaboratively by joining the resources of multiple parties.
  arXiv  Detail & Related papers  (2022-09-02T17:38:03Z)
• Improving the Learnability of Machine Learning APIs by Semi-Automated API Wrapping [0.0]
  We address the challenge of creating APIs that are easy to learn and use, especially by novices. We investigate this problem for skl, a widely used ML API. We identify unused and apparently useless parts of the API that can be eliminated without affecting client programs.
  arXiv  Detail & Related papers  (2022-03-29T12:42:05Z)
• Did the Model Change? Efficiently Assessing Machine Learning API Shifts [24.342984907651505]
  Machine learning (ML) prediction APIs are increasingly widely used. They can change over time due to model updates or retraining. It is often not clear to the user if and how the ML model has changed.
  arXiv  Detail & Related papers  (2021-07-29T17:41:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.