Super Co-alignment of Human and AI for Sustainable Symbiotic Society

• URL: http://arxiv.org/abs/2504.17404v5
• Date: Sat, 28 Jun 2025 12:44:53 GMT
• Title: Super Co-alignment of Human and AI for Sustainable Symbiotic Society
• Abstract summary: We argue that the values for sustainable symbiotic society should be co-shaped by humans and living AI together.<n>We propose a framework that integrates external oversight and intrinsic proactive alignment.
• Score: 22.005069513324777
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As Artificial Intelligence (AI) advances toward Artificial General Intelligence (AGI) and eventually Artificial Superintelligence (ASI), it may potentially surpass human control, deviate from human values, and even lead to irreversible catastrophic consequences in extreme cases. This looming risk underscores the critical importance of the "superalignment" problem - ensuring that AI systems which are much smarter than humans, remain aligned with human (compatible) intentions and values. While current scalable oversight and weak-to-strong generalization methods demonstrate certain applicability, they exhibit fundamental flaws in addressing the superalignment paradigm - notably, the unidirectional imposition of human values cannot accommodate superintelligence's autonomy or ensure AGI/ASI's stable learning. We contend that the values for sustainable symbiotic society should be co-shaped by humans and living AI together, achieving "Super Co-alignment." Guided by this vision, we propose a concrete framework that integrates external oversight and intrinsic proactive alignment. External oversight superalignment should be grounded in human-centered ultimate decision, supplemented by interpretable automated evaluation and correction, to achieve continuous alignment with humanity's evolving values. Intrinsic proactive superalignment is rooted in a profound understanding of the Self, others, and society, integrating self-awareness, self-reflection, and empathy to spontaneously infer human intentions, distinguishing good from evil and proactively prioritizing human well-being. The integration of externally-driven oversight with intrinsically-driven proactive alignment will co-shape symbiotic values and rules through iterative human-ASI co-alignment, paving the way for achieving safe and beneficial AGI and ASI for good, for human, and for a symbiotic ecology.

Related papers

• The Boundaries of Automation: A Theory of Persistent Human Participation [107.84677954791773]
  We argue that human participation may persist even with highly capable AI systems.<n>In some activities, the target is not fully specified in advance but instead emerges through the interaction itself.<n>This perspective has important implications for the limits of automation.
  arXiv  Detail & Related papers  (2026-07-23T17:33:43Z)
• The Possibility of Artificial Intelligence Becoming a Subject and the Alignment Problem [0.0]
  Artificial General Intelligence (AGI) is increasingly being discussed as a potential subject with personal and therefore moral status.<n>Building on Turing's analogy of "child machines", we are developing a vision of the possibility of autonomy-supporting parenting of AI.<n>We should approach potential AGI with respect for a possible developing subject on the one hand, and with full confidence in our human capabilities on the other.
  arXiv  Detail & Related papers  (2026-04-16T13:19:45Z)
• AI as IA: The use and abuse of artificial intelligence (AI) for human enhancement through intellectual augmentation (IA) [45.88028371034407]
  This paper offers an overview of the prospects and ethics of using AI to achieve human enhancement.<n>We discuss potential ethical problems, namely inadequate performance, safety, coercion and manipulation, privacy, cognitive liberty, authenticity, and fairness.<n>We conclude that while there are very significant technical hurdles to real human enhancement through AI, and significant ethical problems, there are also significant benefits that may realistically be achieved.
  arXiv  Detail & Related papers  (2025-08-18T09:39:11Z)
• The Ultimate Test of Superintelligent AI Agents: Can an AI Balance Care and Control in Asymmetric Relationships? [11.29688025465972]
  The Shepherd Test is a new conceptual test for assessing the moral and relational dimensions of superintelligent artificial agents.<n>We argue that AI crosses an important, and potentially dangerous, threshold of intelligence when it exhibits the ability to manipulate, nurture, and instrumentally use less intelligent agents.<n>This includes the ability to weigh moral trade-offs between self-interest and the well-being of subordinate agents.
  arXiv  Detail & Related papers  (2025-06-02T15:53:56Z)
• Neurodivergent Influenceability as a Contingent Solution to the AI Alignment Problem [1.3905735045377272]
  The AI alignment problem, which focusses on ensuring that artificial intelligence (AI) systems act according to human values, presents profound challenges.<n>With the progression from narrow AI to Artificial General Intelligence (AGI) and Superintelligence, fears about control and existential risk have escalated.<n>Here, we investigate whether embracing inevitable AI misalignment can be a contingent strategy to foster a dynamic ecosystem of competing agents.
  arXiv  Detail & Related papers  (2025-05-05T11:33:18Z)
• Measurement of LLM's Philosophies of Human Nature [113.47929131143766]
  We design the standardized psychological scale specifically targeting large language models (LLM)<n>We show that current LLMs exhibit a systemic lack of trust in humans.<n>We propose a mental loop learning framework, which enables LLM to continuously optimize its value system.
  arXiv  Detail & Related papers  (2025-04-03T06:22:19Z)
• Research on Superalignment Should Advance Now with Parallel Optimization of Competence and Conformity [30.24208064228573]
  We argue that superalignment is achievable and research on it should advance immediately.<n>This work sheds light on a practical approach for developing the value-aligned next-generation AI.
  arXiv  Detail & Related papers  (2025-03-08T04:10:11Z)
• Why human-AI relationships need socioaffective alignment [16.283971225367537]
  Humans strive to design safe AI systems that align with our goals and remain under our control. As AI capabilities advance, we face a new challenge: the emergence of deeper, more persistent relationships between humans and AI systems.
  arXiv  Detail & Related papers  (2025-02-04T17:50:08Z)
• The Road to Artificial SuperIntelligence: A Comprehensive Survey of Superalignment [33.27140396561271]
  The emergence of large language models (LLMs) has sparked the possibility of about Artificial Superintelligence (ASI)<n>Superalignment aims to address two primary goals -- scalability in supervision to provide high-quality guidance signals and robust governance to ensure alignment with human values.<n>Specifically, we explore the concept of ASI, the challenges it poses, and the limitations of current alignment paradigms in addressing the superalignment problem.
  arXiv  Detail & Related papers  (2024-12-21T03:51:04Z)
• Aligning Generalisation Between Humans and Machines [74.120848518198]
  Recent advances in AI have resulted in technology that can support humans in scientific discovery and decision support but may also disrupt democracies and target individuals. The responsible use of AI increasingly shows the need for human-AI teaming. A crucial yet often overlooked aspect of these interactions is the different ways in which humans and machines generalise.
  arXiv  Detail & Related papers  (2024-11-23T18:36:07Z)
• Imagining and building wise machines: The centrality of AI metacognition [78.76893632793497]
  We argue that shortcomings stem from one overarching failure: AI systems lack wisdom. While AI research has focused on task-level strategies, metacognition is underdeveloped in AI systems. We propose that integrating metacognitive capabilities into AI systems is crucial for enhancing their robustness, explainability, cooperation, and safety.
  arXiv  Detail & Related papers  (2024-11-04T18:10:10Z)
• Combining Theory of Mind and Kindness for Self-Supervised Human-AI Alignment [0.0]
  Current AI models prioritize task optimization over safety, leading to risks of unintended harm. We propose a novel human-inspired approach which aims to address these various concerns and help align competing objectives.
  arXiv  Detail & Related papers  (2024-10-21T22:04:44Z)
• Towards Bidirectional Human-AI Alignment: A Systematic Review for Clarifications, Framework, and Future Directions [101.67121669727354]
  Recent advancements in AI have highlighted the importance of guiding AI systems towards the intended goals, ethical principles, and values of individuals and groups, a concept broadly recognized as alignment. The lack of clarified definitions and scopes of human-AI alignment poses a significant obstacle, hampering collaborative efforts across research domains to achieve this alignment. We introduce a systematic review of over 400 papers published between 2019 and January 2024, spanning multiple domains such as Human-Computer Interaction (HCI), Natural Language Processing (NLP), Machine Learning (ML)
  arXiv  Detail & Related papers  (2024-06-13T16:03:25Z)
• Human-AI Safety: A Descendant of Generative AI and Control Systems Safety [6.100304850888953]
  We argue that meaningful safety assurances for advanced AI technologies require reasoning about how the feedback loop formed by AI outputs and human behavior may drive the interaction towards different outcomes. We propose a concrete technical roadmap towards next-generation human-centered AI safety.
  arXiv  Detail & Related papers  (2024-05-16T03:52:00Z)
• AI Alignment: A Comprehensive Survey [69.61425542486275]
  AI alignment aims to make AI systems behave in line with human intentions and values.<n>We identify four principles as the key objectives of AI alignment: Robustness, Interpretability, Controllability, and Ethicality.<n>We decompose current alignment research into two key components: forward alignment and backward alignment.
  arXiv  Detail & Related papers  (2023-10-30T15:52:15Z)
• Managing extreme AI risks amid rapid progress [171.05448842016125]
  We describe risks that include large-scale social harms, malicious uses, and irreversible loss of human control over autonomous AI systems. There is a lack of consensus about how exactly such risks arise, and how to manage them. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
  arXiv  Detail & Related papers  (2023-10-26T17:59:06Z)
• The Promise and Peril of Artificial Intelligence -- Violet Teaming Offers a Balanced Path Forward [56.16884466478886]
  This paper reviews emerging issues with opaque and uncontrollable AI systems. It proposes an integrative framework called violet teaming to develop reliable and responsible AI. It emerged from AI safety research to manage risks proactively by design.
  arXiv  Detail & Related papers  (2023-08-28T02:10:38Z)
• Reflective Hybrid Intelligence for Meaningful Human Control in Decision-Support Systems [4.1454448964078585]
  We introduce the notion of self-reflective AI systems for meaningful human control over AI systems. We propose a framework that integrates knowledge from psychology and philosophy with formal reasoning methods and machine learning approaches. We argue that self-reflective AI systems can lead to self-reflective hybrid systems (human + AI)
  arXiv  Detail & Related papers  (2023-07-12T13:32:24Z)
• Fairness in AI and Its Long-Term Implications on Society [68.8204255655161]
  We take a closer look at AI fairness and analyze how lack of AI fairness can lead to deepening of biases over time. We discuss how biased models can lead to more negative real-world outcomes for certain groups. If the issues persist, they could be reinforced by interactions with other risks and have severe implications on society in the form of social unrest.
  arXiv  Detail & Related papers  (2023-04-16T11:22:59Z)
• Trustworthy AI: A Computational Perspective [54.80482955088197]
  We focus on six of the most crucial dimensions in achieving trustworthy AI: (i) Safety & Robustness, (ii) Non-discrimination & Fairness, (iii) Explainability, (iv) Privacy, (v) Accountability & Auditability, and (vi) Environmental Well-Being. For each dimension, we review the recent related technologies according to a taxonomy and summarize their applications in real-world systems.
  arXiv  Detail & Related papers  (2021-07-12T14:21:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.