Federated Item Response Theory Models

• URL: http://arxiv.org/abs/2506.21744v1
• Date: Thu, 26 Jun 2025 20:01:18 GMT
• Title: Federated Item Response Theory Models
• Abstract summary: We propose a novel framework, Federated Item Response Theory (IRT), to enable estimating traditional IRT models with additional privacy.<n>Our experiments confirm that FedIRT achieves statistical accuracy similar to standard IRT estimation using popular R packages.<n>This new framework extends IRT's applicability to distributed settings, such as multi-school assessments, without sacrificing accuracy or security.
• Score: 8.125608919874074
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Item Response Theory (IRT) models have been widely used to estimate respondents' latent abilities and calibrate items' difficulty. Traditional IRT estimation requires all individual raw response data to be centralized in one place, thus potentially causing privacy issues. Federated learning is an emerging field in computer science and machine learning with added features of privacy protection and distributed computing. To integrate the advances from federated learning with modern psychometrics, we propose a novel framework, Federated Item Response Theory (IRT), to enable estimating traditional IRT models with additional privacy, allowing estimation in a distributed manner without losing estimation accuracy. Our numerical experiments confirm that FedIRT achieves statistical accuracy similar to standard IRT estimation using popular R packages, while offering critical advantages: privacy protection and reduced communication costs. We also validate FedIRT's utility through a real-world exam dataset, demonstrating its effectiveness in realistic educational contexts. This new framework extends IRT's applicability to distributed settings, such as multi-school assessments, without sacrificing accuracy or security. To support practical adoption, we provide an open-ource R package, FedIRT, implementing the framework for the two-parameter logistic (2PL) and partial credit models (PCM).

Related papers

• Prediction-powered Inference by Mixture of Experts [14.293038907072598]
  We introduce an MOE-powered semi-supervised inference framework built upon prediction-powered inference (PPI)<n>Motivated by the variance reduction principle underlying PPI, the proposed framework seeks the mixture of experts that achieves the smallest possible variance.<n>The framework is flexible and applies to mean estimation, linear regression, quantile estimation, and general M-estimation.
  arXiv  Detail & Related papers  (2026-04-30T14:08:17Z)
• RADAR: Revealing Asymmetric Development of Abilities in MLLM Pre-training [59.493415006017635]
  Pre-trained Multi-modal Large Language Models (MLLMs) provide a knowledge-rich foundation for post-training.<n>Current evaluation relies on testing after supervised fine-tuning, which introduces laborious additional training and autoregressive decoding costs.<n>We propose RADAR, an efficient ability-centric evaluation framework for Revealing Asymmetric Development of Abilities in MLLM pRe-training.
  arXiv  Detail & Related papers  (2026-02-13T12:56:31Z)
• Adaptive Personalized Federated Learning via Multi-task Averaging of Kernel Mean Embeddings [16.820009380421368]
  Federated learning enables a collection of agents to collaboratively learn individual models without sharing raw data.<n>We propose a new PFL approach in which each agent optimize a weighted combination of all agents' empirical risks, with the weights learned from data rather than specified a priori.
  arXiv  Detail & Related papers  (2026-02-13T07:49:03Z)
• Federated Computation of ROC and PR Curves [8.64427265159929]
  In Federated Learning (FL) scenarios, where data is distributed across multiple clients, computing ROC and PR curves is challenging due to privacy and communication constraints.<n>We propose a novel method for approximating ROC and PR curves in a federated setting by estimating quantiles of the prediction score distribution under distributed differential privacy.
  arXiv  Detail & Related papers  (2025-10-06T16:16:46Z)
• Testing Fairness with Utility Tradeoffs: A Wasserstein Projection Approach [6.378410364292642]
  We propose a statistical hypothesis testing framework that jointly evaluates approximate fairness and utility.<n>Our framework builds on the strong demographic parity criterion and incorporates a utility measure motivated by the potential outcomes framework.<n>We show that the test is computationally tractable, interpretable, broadly applicable across machine learning models, and extendable to more general settings.
  arXiv  Detail & Related papers  (2025-05-16T20:29:06Z)
• ExpertRAG: Efficient RAG with Mixture of Experts -- Optimizing Context Retrieval for Adaptive LLM Responses [0.0]
  ExpertRAG is a novel theoretical framework that integrates Mixture-of-Experts (MoE) architectures with Retrieval Augmented Generation (RAG)<n>We propose a dynamic retrieval gating mechanism coupled with expert routing, enabling the model to selectively consult an external knowledge store or rely on specialized internal experts.<n>We derive formulae to quantify the expected computational cost savings from selective retrieval and the capacity gains from sparse expert utilization.
  arXiv  Detail & Related papers  (2025-03-23T17:26:23Z)
• Privacy-Preserved Automated Scoring using Federated Learning for Educational Research [1.2556373621040728]
  We propose a federated learning (FL) framework for automated scoring of educational assessments.<n>We benchmark our model against two state-of-the-art FL methods and a centralized learning baseline.<n>Results show that our model achieves the highest accuracy (94.5%) among FL approaches.
  arXiv  Detail & Related papers  (2025-03-12T19:06:25Z)
• Reconciling Privacy and Explainability in High-Stakes: A Systematic Inquiry [0.0]
  Deep learning's preponderance across scientific domains has reshaped high-stakes decision-making.<n>This paper examines the complexities of combining Right-to-Privacy (RTP) and Right-to-Explanation (RTE)
  arXiv  Detail & Related papers  (2024-12-30T08:43:28Z)
• Reliable Confidence Intervals for Information Retrieval Evaluation Using Generative A.I [39.92942310783174]
  Large language models (LLMs) can generate relevance annotations at an enormous scale with relatively small computational costs. We propose two methods based on prediction-powered inference and conformal risk control. Our experimental results show that our CIs accurately capture both the variance and bias in evaluation.
  arXiv  Detail & Related papers  (2024-07-02T17:44:00Z)
• Federated Learning with Projected Trajectory Regularization [65.6266768678291]
  Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across the clients. We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
  arXiv  Detail & Related papers  (2023-12-22T02:12:08Z)
• f-FERM: A Scalable Framework for Robust Fair Empirical Risk Minimization [9.591164070876689]
  This paper presents a unified optimization framework for fair empirical risk based on f-divergence measures (f-FERM) In addition, our experiments demonstrate the superiority of fairness-accuracy tradeoffs offered by f-FERM for almost all batch sizes. Our extension is based on a distributionally robust optimization reformulation of f-FERM objective under $L_p$ norms as uncertainty sets.
  arXiv  Detail & Related papers  (2023-12-06T03:14:16Z)
• Improved Policy Evaluation for Randomized Trials of Algorithmic Resource Allocation [54.72195809248172]
  We present a new estimator leveraging our proposed novel concept, that involves retrospective reshuffling of participants across experimental arms at the end of an RCT. We prove theoretically that such an estimator is more accurate than common estimators based on sample means.
  arXiv  Detail & Related papers  (2023-02-06T05:17:22Z)
• Differential privacy and robust statistics in high dimensions [49.50869296871643]
  High-dimensional Propose-Test-Release (HPTR) builds upon three crucial components: the exponential mechanism, robust statistics, and the Propose-Test-Release mechanism. We show that HPTR nearly achieves the optimal sample complexity under several scenarios studied in the literature.
  arXiv  Detail & Related papers  (2021-11-12T06:36:40Z)
• Tight Mutual Information Estimation With Contrastive Fenchel-Legendre Optimization [69.07420650261649]
  We introduce a novel, simple, and powerful contrastive MI estimator named as FLO. Empirically, our FLO estimator overcomes the limitations of its predecessors and learns more efficiently. The utility of FLO is verified using an extensive set of benchmarks, which also reveals the trade-offs in practical MI estimation.
  arXiv  Detail & Related papers  (2021-07-02T15:20:41Z)
• Confidence Estimation via Auxiliary Models [47.08749569008467]
  We introduce a novel target criterion for model confidence, namely the true class probability ( TCP) We show that TCP offers better properties for confidence estimation than standard maximum class probability (MCP)
  arXiv  Detail & Related papers  (2020-12-11T17:21:12Z)
• Edge-assisted Democratized Learning Towards Federated Analytics [67.44078999945722]
  We show the hierarchical learning structure of the proposed edge-assisted democratized learning mechanism, namely Edge-DemLearn. We also validate Edge-DemLearn as a flexible model training mechanism to build a distributed control and aggregation methodology in regions.
  arXiv  Detail & Related papers  (2020-12-01T11:46:03Z)
• Stochastic-Sign SGD for Federated Learning with Theoretical Guarantees [49.91477656517431]
  Quantization-based solvers have been widely adopted in Federated Learning (FL) No existing methods enjoy all the aforementioned properties. We propose an intuitively-simple yet theoretically-simple method based on SIGNSGD to bridge the gap.
  arXiv  Detail & Related papers  (2020-02-25T15:12:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.