CoMAI: A Collaborative Multi-Agent Framework for Robust and Equitable Interview Evaluation

• URL: http://arxiv.org/abs/2603.16215v1
• Date: Tue, 17 Mar 2026 07:44:12 GMT
• Title: CoMAI: A Collaborative Multi-Agent Framework for Robust and Equitable Interview Evaluation
• Abstract summary: CoMAI is a general-purpose multi-agent interview framework designed for diverse assessment scenarios.<n>The system comprises four agents specialized in question generation, security, scoring, and summarization.<n> Experimental results demonstrate that CoMAI achieved 90.47% accuracy, 83.33% recall, and 84.41% candidate satisfaction.
• Score: 4.938120508930052
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Ensuring robust and fair interview assessment remains a key challenge in AI-driven evaluation. This paper presents CoMAI, a general-purpose multi-agent interview framework designed for diverse assessment scenarios. In contrast to monolithic single-agent systems based on large language models (LLMs), CoMAI employs a modular task-decomposition architecture coordinated through a centralized finite-state machine. The system comprises four agents specialized in question generation, security, scoring, and summarization. These agents work collaboratively to provide multi-layered security defenses against prompt injection, support multidimensional evaluation with adaptive difficulty adjustment, and enable rubric-based structured scoring that reduces subjective bias. Experimental results demonstrate that CoMAI achieved 90.47% accuracy, 83.33% recall, and 84.41% candidate satisfaction. These results highlight CoMAI as a robust, fair, and interpretable paradigm for AI-driven interview assessment.

Related papers

• AgentBeats: Agentifying Agent Assessment for Openness, Standardization, and Reproducibility [104.46861849039357]
  Agent systems are advancing quickly across domains, but their evaluation remains fragmented.<n>The root problem is the lack of an open, agent-agnostic assessment interface.<n>We advocate Agentified Agent Assessment (AAA), where evaluation is performed by judge agents and all participants interact through standardized protocols.
  arXiv  Detail & Related papers  (2026-06-11T17:23:54Z)
• Plan First, Judge Later, Run Better: A DMAIC-Inspired Agentic System for Industrial Anomaly Detection [51.90293101535452]
  We propose DMAIC-IAD (DMAIC-inspired Agentic Industrial Anomaly Detection), a "Plan First, Judge Later" multi-agent system.<n>We show DMAIC-IAD improves average detection performance over applicable agentic baselines by 37.76%.
  arXiv  Detail & Related papers  (2026-06-03T08:38:14Z)
• Pitfalls in Evaluating Interpretability Agents [91.49742416116635]
  We build an agentic system in which a research agent iteratively designs experiments and refines hypotheses.<n>Our work demonstrates fundamental challenges in evaluating complex automated interpretability systems.
  arXiv  Detail & Related papers  (2026-03-20T16:27:17Z)
• CollabEval: Enhancing LLM-as-a-Judge via Multi-Agent Collaboration [8.457834313970165]
  Large Language Models (LLMs) have revolutionized AI-generated content evaluation, with the LLM-as-a-Judge paradigm becoming increasingly popular.<n>We propose CollabEval, a novel multi-agent evaluation framework that implements a three-phase Collaborative Evaluation process.<n>Our experiments demonstrate that CollabEval consistently outperforms single-LLM approaches across multiple dimensions.
  arXiv  Detail & Related papers  (2026-03-01T08:45:13Z)
• AirCopBench: A Benchmark for Multi-drone Collaborative Embodied Perception and Reasoning [33.49390843688711]
  Multimodal Large Language Models (MLLMs) have shown promise in single-agent vision tasks.<n>AirCopBench is the first comprehensive benchmark designed to evaluate MLLMs in embodied aerial perception.
  arXiv  Detail & Related papers  (2025-11-14T07:23:05Z)
• A Rigorous Benchmark with Multidimensional Evaluation for Deep Research Agents: From Answers to Reports [24.09178055088843]
  Deep Research Agents (DRAs) exhibit the capabilities for task decomposition, cross-source retrieval, multi-stage reasoning, and structured output.<n>This paper introduces a rigorous benchmark and a multidimensional evaluation framework tailored to DRAs and report-style responses.<n>The framework enables comprehensive evaluation of long-form reports generated by DRAs, incorporating integrated scoring metrics for semantic quality, topical focus, and retrieval trustworthiness.
  arXiv  Detail & Related papers  (2025-10-02T16:40:02Z)
• A Multi-To-One Interview Paradigm for Efficient MLLM Evaluation [63.76972456980632]
  We propose a multi-to-one interview paradigm for efficient MLLM evaluation.<n>Our framework consists of (i) a two-stage interview strategy with pre-interview and formal interview phases, (ii) dynamic adjustment of weights to ensure fairness, and (iii) an adaptive mechanism for question difficulty-level chosen.
  arXiv  Detail & Related papers  (2025-09-18T12:07:40Z)
• An Automated Multi-modal Evaluation Framework for Mobile Intelligent Assistants Based on Large Language Models and Multi-Agent Collaboration [5.630646705589818]
  This paper proposes an automated multi-modal evaluation framework based on large language models and multi-agent collaboration.<n>The framework employs a three-tier agent architecture consisting of interaction evaluation agents, semantic verification agents, and experience decision agents.
  arXiv  Detail & Related papers  (2025-08-13T05:40:34Z)
• Auto-Eval Judge: Towards a General Agentic Framework for Task Completion Evaluation [4.08768677009363]
  We propose a generalizable, modular framework for evaluating agent task completion independent of the task domain.<n>We validate our framework by evaluating the Magentic-One Actor Agent on two benchmarks, GAIA and BigCodeBench.<n>Our Judge Agent predicts task success with closer agreement to human evaluations, achieving 4.76% and 10.52% higher alignment accuracy, respectively.
  arXiv  Detail & Related papers  (2025-08-07T15:39:48Z)
• Establishing Best Practices for Building Rigorous Agentic Benchmarks [94.69724201080155]
  We show that many agentic benchmarks have issues in task setup or reward design.<n>Such issues can lead to under- or overestimation of agents' performance by up to 100% in relative terms.<n>We introduce the Agentic Benchmark Checklist (ABC), a set of guidelines that we synthesized from our benchmark-building experience.
  arXiv  Detail & Related papers  (2025-07-03T17:35:31Z)
• MTR-Bench: A Comprehensive Benchmark for Multi-Turn Reasoning Evaluation [56.87891213797931]
  We present MTR-Bench for Large Language Models' Multi-Turn Reasoning evaluation.<n>Comprising 4 classes, 40 tasks, and 3600 instances, MTR-Bench covers diverse reasoning capabilities.<n>MTR-Bench features fully-automated framework spanning both dataset constructions and model evaluations.
  arXiv  Detail & Related papers  (2025-05-21T17:59:12Z)
• AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents [74.16170899755281]
  We introduce AgentBoard, a pioneering comprehensive benchmark and accompanied open-source evaluation framework tailored to analytical evaluation of LLM agents.<n>AgentBoard offers a fine-grained progress rate metric that captures incremental advancements as well as a comprehensive evaluation toolkit.<n>This not only sheds light on the capabilities and limitations of LLM agents but also propels the interpretability of their performance to the forefront.
  arXiv  Detail & Related papers  (2024-01-24T01:51:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.