LM Fight Arena: Benchmarking Large Multimodal Models via Game Competition

• URL: http://arxiv.org/abs/2510.08928v1
• Date: Fri, 10 Oct 2025 02:19:21 GMT
• Title: LM Fight Arena: Benchmarking Large Multimodal Models via Game Competition
• Authors: Yushuo Zheng, Zicheng Zhang, Xiongkuo Min, Huiyu Duan, Guangtao Zhai,
• Abstract summary: We introduce LM Fight Arena, a novel framework that evaluates large multimodal models in Mortal Kombat II.<n>Unlike static evaluations, LM Fight Arena provides a fully automated, reproducible, and objective assessment of an LMM's strategic reasoning capabilities.
• Score: 104.81487689011341
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing benchmarks for large multimodal models (LMMs) often fail to capture their performance in real-time, adversarial environments. We introduce LM Fight Arena (Large Model Fight Arena), a novel framework that evaluates LMMs by pitting them against each other in the classic fighting game Mortal Kombat II, a task requiring rapid visual understanding and tactical, sequential decision-making. In a controlled tournament, we test six leading open- and closed-source models, where each agent operates controlling the same character to ensure a fair comparison. The models are prompted to interpret game frames and state data to select their next actions. Unlike static evaluations, LM Fight Arena provides a fully automated, reproducible, and objective assessment of an LMM's strategic reasoning capabilities in a dynamic setting. This work introduces a challenging and engaging benchmark that bridges the gap between AI evaluation and interactive entertainment.

Related papers

• BotzoneBench: Scalable LLM Evaluation via Graded AI Anchors [9.224594551677374]
  Large Language Models (LLMs) are increasingly deployed in interactive environments requiring strategic decision-making.<n>Recent game-based evaluations employ LLM-vs-LLM tournaments that produce relative rankings dependent on transient model pools.<n>Here we show that anchoring LLM evaluation to fixed hierarchies of skill-calibrated game Artificial Intelligence (AI) enables linear-time absolute skill measurement.
  arXiv  Detail & Related papers  (2026-01-22T13:15:08Z)
• LLM CHESS: Benchmarking Reasoning and Instruction-Following in LLMs through Chess [30.797553771114746]
  We introduce LLM CHESS, an evaluation framework designed to probe the generalization of reasoning and instruction-following abilities in large language models (LLMs)<n>We rank over 50 open and closed source models by playing against a random opponent using a range of behavioral metrics, including move quality, move legality, hallucinated actions, and game duration.<n>For a subset of top reasoning models, we derive an Elo estimate by playing against a chess engine with variably configured skill, which allows for comparisons between models in an easily understandable way.
  arXiv  Detail & Related papers  (2025-12-01T18:51:08Z)
• PillagerBench: Benchmarking LLM-Based Agents in Competitive Minecraft Team Environments [48.892997022500765]
  We introduce PillagerBench, a framework for evaluating multi-agent systems in real-time competitive team-vs-team scenarios in Minecraft.<n>We also propose TactiCrafter, an LLM-based multi-agent system that facilitates teamwork through human-readable tactics.<n>Our evaluation demonstrates that TactiCrafter outperforms baseline approaches and showcases adaptive learning through self-play.
  arXiv  Detail & Related papers  (2025-09-07T22:51:12Z)
• Who is a Better Player: LLM against LLM [53.46608216197315]
  We propose an adversarial benchmarking framework to assess the comprehensive performance of Large Language Models (LLMs) through board games competition.<n>We introduce Qi Town, a specialized evaluation platform that supports 5 widely played games and involves 20 LLM-driven players.
  arXiv  Detail & Related papers  (2025-08-05T06:41:47Z)
• SPARTA ALIGNMENT: Collectively Aligning Multiple Language Models through Combat [73.529925653031]
  We propose SPARTA ALIGNMENT, an algorithm to collectively align multiple LLMs through competition and combat.<n>For each iteration, one instruction and two models are selected for a duel, the other models evaluate the two responses, and their evaluation scores are aggregated through a adapted elo-ranking based reputation system.<n>The peer-evaluated combat results then become preference pairs where the winning response is preferred over the losing one, and all models learn from these preferences at the end of each iteration.
  arXiv  Detail & Related papers  (2025-06-05T07:51:23Z)
• Decentralized Arena: Towards Democratic and Scalable Automatic Evaluation of Language Models [66.51871176061195]
  Decentralized Arena (dearena) is a fully automated framework leveraging collective intelligence from all large language models to evaluate each other.<n> dearena attains up to 97% correlation with human judgements, while significantly reducing the cost.
  arXiv  Detail & Related papers  (2025-05-19T07:34:25Z)
• TMGBench: A Systematic Game Benchmark for Evaluating Strategic Reasoning Abilities of LLMs [45.12542636218608]
  We propose TMGBench, characterized by comprehensive game type coverage, diverse scenarios and flexible game organization.<n>Specifically, we incorporate all 144 game types summarized by the Robinson-Goforth topology of 2x2 games, constructed as classic games in our benchmark.<n>To provide a sustainable evaluation framework adaptable to increasingly powerful LLMs, we treat the aforementioned games as atomic units.
  arXiv  Detail & Related papers  (2024-10-14T13:15:34Z)
• FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning [25.857375787748715]
  We present FightLadder, a real-time fighting game platform, to empower competitive MARL research. We provide implementations of state-of-the-art MARL algorithms for competitive games, as well as a set of evaluation metrics. We demonstrate the feasibility of this platform by training a general agent that consistently defeats 12 built-in characters in single-player mode.
  arXiv  Detail & Related papers  (2024-06-04T08:04:23Z)
• L2E: Learning to Exploit Your Opponent [66.66334543946672]
  We propose a novel Learning to Exploit framework for implicit opponent modeling. L2E acquires the ability to exploit opponents by a few interactions with different opponents during training. We propose a novel opponent strategy generation algorithm that produces effective opponents for training automatically.
  arXiv  Detail & Related papers  (2021-02-18T14:27:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.