TOM: A Development Platform For Wearable Intelligent Assistants

• URL: http://arxiv.org/abs/2407.15523v1
• Date: Mon, 22 Jul 2024 10:20:02 GMT
• Title: TOM: A Development Platform For Wearable Intelligent Assistants
• Authors: Nuwan Janaka, Shengdong Zhao, David Hsu, Sherisse Tan Jing Wen, Koh Chun Keat,
• Abstract summary: We introduce TOM, a conceptual architecture and software platform to support the development of intelligent wearable assistants. TOM supports the recording and analysis of user interactions, integration of new devices, and the provision of assistance for various activities. We showcase several proof-of-concept assistive services and discuss the challenges involved in developing such services.
• Score: 23.072788336462683
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Advanced digital assistants can significantly enhance task performance, reduce user burden, and provide personalized guidance to improve users' abilities. However, the development of such intelligent digital assistants presents a formidable challenge. To address this, we introduce TOM, a conceptual architecture and software platform (https://github.com/TOM-Platform) designed to support the development of intelligent wearable assistants that are contextually aware of both the user and the environment. This system was developed collaboratively with AR/MR researchers, HCI researchers, AI/Robotic researchers, and software developers, and it continues to evolve to meet the diverse requirements of these stakeholders. TOM facilitates the creation of intelligent assistive AR applications for daily activities and supports the recording and analysis of user interactions, integration of new devices, and the provision of assistance for various activities. Additionally, we showcase several proof-of-concept assistive services and discuss the challenges involved in developing such services.

Related papers

• SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation [89.24729958546168]
  We present SPA-Bench, a comprehensive SmartPhone Agent Benchmark designed to evaluate (M)LLM-based agents. SPA-Bench offers three key contributions: A diverse set of tasks covering system and third-party apps in both English and Chinese, focusing on features commonly used in daily routines. A novel evaluation pipeline that automatically assesses agent performance across multiple dimensions, encompassing seven metrics related to task completion and resource consumption.
  arXiv  Detail & Related papers  (2024-10-19T17:28:48Z)
• AssistantX: An LLM-Powered Proactive Assistant in Collaborative Human-Populated Environment [15.475084260674384]
  AssistantX is a proactive assistant designed to operate autonomously in a physical office environment. Unlike conventional service robots, AssistantX leverages a novel multi-agent architecture, PPDR4X, which provides advanced inference capabilities. Our evaluation highlights the architecture's effectiveness, showing that AssistantX can respond to clear instructions, actively retrieve supplementary information from memory, and proactively seek collaboration from team members to ensure successful task completion.
  arXiv  Detail & Related papers  (2024-09-26T09:06:56Z)
• The Ethics of Advanced AI Assistants [53.89899371095332]
  This paper focuses on the opportunities and the ethical and societal risks posed by advanced AI assistants. We define advanced AI assistants as artificial agents with natural language interfaces, whose function is to plan and execute sequences of actions on behalf of a user. We consider the deployment of advanced assistants at a societal scale, focusing on cooperation, equity and access, misinformation, economic impact, the environment and how best to evaluate advanced AI assistants.
  arXiv  Detail & Related papers  (2024-04-24T23:18:46Z)
• Investigating Remote Hands-On Assistance for Collaborative Development of Embedded Systems [3.9877913211157843]
  This study investigates the potential for remote support tools specifically designed for embedded systems development. Through interviews with 12 experienced embedded systems developers, we explored their existing remote work practices, challenges, and requirements. Our findings highlight the scenarios and strategies employed in remote work, the specific support needs, and the challenges related to information exchange, coordination, and execution.
  arXiv  Detail & Related papers  (2024-04-24T20:28:20Z)
• Bridging Gaps, Building Futures: Advancing Software Developer Diversity and Inclusion Through Future-Oriented Research [50.545824691484796]
  We present insights from SE researchers and practitioners on challenges and solutions regarding diversity and inclusion in SE. We share potential utopian and dystopian visions of the future and provide future research directions and implications for academia and industry.
  arXiv  Detail & Related papers  (2024-04-10T16:18:11Z)
• A Design Space for Intelligent and Interactive Writing Assistants [55.9780345526642]
  We explore five aspects of writing assistants: task, user, technology, interaction, and ecosystem. Within each aspect, we define dimensions (i.e., fundamental components of an aspect) and codes (i.e., potential options for each dimension) by systematically reviewing 115 papers. Our design space aims to offer researchers and designers a practical tool to navigate, comprehend, and compare the various possibilities of writing assistants.
  arXiv  Detail & Related papers  (2024-03-21T04:03:16Z)
• AgentScope: A Flexible yet Robust Multi-Agent Platform [66.64116117163755]
  AgentScope is a developer-centric multi-agent platform with message exchange as its core communication mechanism. The abundant syntactic tools, built-in agents and service functions, user-friendly interfaces for application demonstration and utility monitor, zero-code programming workstation, and automatic prompt tuning mechanism significantly lower the barriers to both development and deployment.
  arXiv  Detail & Related papers  (2024-02-21T04:11:28Z)
• New Interaction Paradigm for Complex EDA Software Leveraging GPT [4.397804186778722]
  An artificial intelligence (AI) interaction assist plugin for EDA software named SmartonAl is developed here. SmartonAI is inspired by the HuggingGPT framework and employs large language models, such as GPT and BERT, to facilitate task planning and execution. Our preliminary results demonstrate that SmartonAI can significantly streamline the PCB design process by simplifying complex commands into intuitive language-based interactions.
  arXiv  Detail & Related papers  (2023-07-27T09:53:02Z)
• A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challenges [23.467373994306524]
  In practice, developers do not accept AI programming assistants' initial suggestions at a high frequency. To understand developers' practices while using these tools, we administered a survey to a large population of developers. We found that developers are most motivated to use AI programming assistants because they help developers reduce key-strokes, finish programming tasks quickly, and recall syntax. We also found the most important reasons why developers do not use these tools are because these tools do not output code that addresses certain functional or non-functional requirements.
  arXiv  Detail & Related papers  (2023-03-30T03:21:53Z)
• Towards Human-Bot Collaborative Software Architecting with ChatGPT [7.50312929275194]
  Software Development Bots (DevBots) trained on large language models can help synergise architects' knowledge with artificially intelligent decision support. ChatGPT is a disruptive technology not primarily introduced for software engineering. We detail a case study that involves collaboration between a novice software architect and ChatGPT for architectural analysis, synthesis, and evaluation of a services-driven software application.
  arXiv  Detail & Related papers  (2023-02-26T16:32:16Z)
• Towards an AI assistant for human grid operators [59.535699822923]
  Power systems are becoming more complex to operate in the digital age. Real-time decision-making is getting more challenging as the human operator has to deal with more information. There is a great need for rethinking the human-machine interface under more unified and interactive frameworks.
  arXiv  Detail & Related papers  (2020-12-03T16:12:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.