Social Intelligence Data Infrastructure: Structuring the Present and Navigating the Future

• URL: http://arxiv.org/abs/2403.14659v1
• Date: Wed, 28 Feb 2024 00:22:42 GMT
• Title: Social Intelligence Data Infrastructure: Structuring the Present and Navigating the Future
• Authors: Minzhi Li, Weiyan Shi, Caleb Ziems, Diyi Yang,
• Abstract summary: We build a Social AI Data Infrastructure, which consists of a comprehensive social AI taxonomy and a data library of 480 NLP datasets. Our infrastructure allows us to analyze existing dataset efforts, and also evaluate language models' performance in different social intelligence aspects. We show there is a need for multifaceted datasets, increased diversity in language and culture, more long-tailed social situations, and more interactive data in future social intelligence data efforts.
• Score: 59.78608958395464
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As Natural Language Processing (NLP) systems become increasingly integrated into human social life, these technologies will need to increasingly rely on social intelligence. Although there are many valuable datasets that benchmark isolated dimensions of social intelligence, there does not yet exist any body of work to join these threads into a cohesive subfield in which researchers can quickly identify research gaps and future directions. Towards this goal, we build a Social AI Data Infrastructure, which consists of a comprehensive social AI taxonomy and a data library of 480 NLP datasets. Our infrastructure allows us to analyze existing dataset efforts, and also evaluate language models' performance in different social intelligence aspects. Our analyses demonstrate its utility in enabling a thorough understanding of current data landscape and providing a holistic perspective on potential directions for future dataset development. We show there is a need for multifaceted datasets, increased diversity in language and culture, more long-tailed social situations, and more interactive data in future social intelligence data efforts.

Related papers

• Leveraging GPT for the Generation of Multi-Platform Social Media Datasets for Research [0.0]
  Social media datasets are essential for research on disinformation, influence operations, social sensing, hate speech detection, cyberbullying, and other significant topics. Access to these datasets is often restricted due to costs and platform regulations. This paper explores the potential of large language models to create lexically and semantically relevant social media datasets across multiple platforms.
  arXiv  Detail & Related papers  (2024-07-11T09:12:39Z)
• Socialized Learning: A Survey of the Paradigm Shift for Edge Intelligence in Networked Systems [62.252355444948904]
  This paper presents the findings of a literature review on the integration of edge intelligence (EI) and socialized learning (SL) SL is a learning paradigm predicated on social principles and behaviors, aimed at amplifying the collaborative capacity and collective intelligence of agents. We elaborate on three integrated components: socialized architecture, socialized training, and socialized inference, analyzing their strengths and weaknesses.
  arXiv  Detail & Related papers  (2024-04-20T11:07:29Z)
• Advancing Social Intelligence in AI Agents: Technical Challenges and Open Questions [67.60397632819202]
  Building socially-intelligent AI agents (Social-AI) is a multidisciplinary, multimodal research goal. We identify a set of underlying technical challenges and open questions for researchers across computing communities to advance Social-AI.
  arXiv  Detail & Related papers  (2024-04-17T02:57:42Z)
• Social-LLM: Modeling User Behavior at Scale using Language Models and Social Network Data [13.660150473547766]
  We introduce a novel approach tailored for modeling social network data in user detection tasks. Our method integrates localized social network interactions with the capabilities of large language models. We conduct a thorough evaluation of our method across seven real-world social network datasets.
  arXiv  Detail & Related papers  (2023-12-31T05:13:13Z)
• DeSIQ: Towards an Unbiased, Challenging Benchmark for Social Intelligence Understanding [60.84356161106069]
  We study the soundness of Social-IQ, a dataset of multiple-choice questions on videos of complex social interactions. Our analysis reveals that Social-IQ contains substantial biases, which can be exploited by a moderately strong language model. We introduce DeSIQ, a new challenging dataset, constructed by applying simple perturbations to Social-IQ.
  arXiv  Detail & Related papers  (2023-10-24T06:21:34Z)
• SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents [107.4138224020773]
  We present SOTOPIA, an open-ended environment to simulate complex social interactions between artificial agents and humans. In our environment, agents role-play and interact under a wide variety of scenarios; they coordinate, collaborate, exchange, and compete with each other to achieve complex social goals. We find that GPT-4 achieves a significantly lower goal completion rate than humans and struggles to exhibit social commonsense reasoning and strategic communication skills.
  arXiv  Detail & Related papers  (2023-10-18T02:27:01Z)
• Data-centric Artificial Intelligence: A Survey [47.24049907785989]
  Recently, the role of data in AI has been significantly magnified, giving rise to the emerging concept of data-centric AI. In this survey, we discuss the necessity of data-centric AI, followed by a holistic view of three general data-centric goals. We believe this is the first comprehensive survey that provides a global view of a spectrum of tasks across various stages of the data lifecycle.
  arXiv  Detail & Related papers  (2023-03-17T17:44:56Z)
• Embodied AI-Driven Operation of Smart Cities: A Concise Review [3.441021278275805]
  Embodied AI focuses on learning through interaction with the surrounding environment. We will go through its definitions, its characteristics, and its current achievements along with different algorithms, approaches, and solutions. We will then explore all the available simulators and 3D interactable databases that will make the research in this area feasible.
  arXiv  Detail & Related papers  (2021-08-22T19:14:59Z)
• SocialAI: Benchmarking Socio-Cognitive Abilities in Deep Reinforcement Learning Agents [23.719833581321033]
  Building embodied autonomous agents capable of participating in social interactions with humans is one of the main challenges in AI. We argue that aiming towards human-level AI requires a broader set of key social skills. We present SocialAI, a benchmark to assess the acquisition of social skills of DRL agents.
  arXiv  Detail & Related papers  (2021-07-02T10:39:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.