BioArc: Discovering Optimal Neural Architectures for Biological Foundation Models

• URL: http://arxiv.org/abs/2512.00283v2
• Date: Tue, 02 Dec 2025 14:46:22 GMT
• Title: BioArc: Discovering Optimal Neural Architectures for Biological Foundation Models
• Authors: Yi Fang, Haoran Xu, Jiaxin Han, Sirui Ding, Yizhi Wang, Yue Wang, Xuan Wang,
• Abstract summary: Foundation models have revolutionized various fields such as natural language processing (NLP) and computer vision (CV)<n>We introduce BioArc, a novel framework designed to move beyond intuition-driven architecture design towards principled, automated architecture discovery for biological foundation models.<n>Our work provides a foundational resource and a principled methodology to guide the creation of the next generation of task-specific and foundation models for biology.
• Score: 31.218090448573776
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Foundation models have revolutionized various fields such as natural language processing (NLP) and computer vision (CV). While efforts have been made to transfer the success of the foundation models in general AI domains to biology, existing works focus on directly adopting the existing foundation model architectures from general machine learning domains without a systematic design considering the unique physicochemical and structural properties of each biological data modality. This leads to suboptimal performance, as these repurposed architectures struggle to capture the long-range dependencies, sparse information, and complex underlying ``grammars'' inherent to biological data. To address this gap, we introduce BioArc, a novel framework designed to move beyond intuition-driven architecture design towards principled, automated architecture discovery for biological foundation models. Leveraging Neural Architecture Search (NAS), BioArc systematically explores a vast architecture design space, evaluating architectures across multiple biological modalities while rigorously analyzing the interplay between architecture, tokenization, and training strategies. This large-scale analysis identifies novel, high-performance architectures, allowing us to distill a set of empirical design principles to guide future model development. Furthermore, to make the best of this set of discovered principled architectures, we propose and compare several architecture prediction methods that effectively and efficiently predict optimal architectures for new biological tasks. Overall, our work provides a foundational resource and a principled methodology to guide the creation of the next generation of task-specific and foundation models for biology.

Related papers

• Breaking Robustness Barriers in Cognitive Diagnosis: A One-Shot Neural Architecture Search Perspective [19.30893604363489]
  We propose One-Shot neural architecture search method for Cognitive Diagnosis.<n> OSCD operates through two distinct stages: training and searching.<n>In searching stage, we formulate the optimal architecture search under heterogeneous noise scenarios.
  arXiv  Detail & Related papers  (2026-01-08T13:17:40Z)
• Copresheaf Topological Neural Networks: A Generalized Deep Learning Framework [16.903981913294103]
  We introduce copresheaf topological neural networks (CTNNs)<n>CTNNs are a powerful unifying framework that encapsulates a wide spectrum of deep learning architectures.<n>We show that CTNNs consistently outperform conventional baselines in tasks requiring hierarchical or localized sensitivity.
  arXiv  Detail & Related papers  (2025-05-27T14:28:50Z)
• Graph Foundation Models: A Comprehensive Survey [66.74249119139661]
  Graph Foundation Models (GFMs) aim to bring scalable, general-purpose intelligence to structured data.<n>This survey provides a comprehensive overview of GFMs, unifying diverse efforts under a modular framework.<n>GFMs are poised to become foundational infrastructure for open-ended reasoning over structured data.
  arXiv  Detail & Related papers  (2025-05-21T05:08:00Z)
• Exploring Synergistic Ensemble Learning: Uniting CNNs, MLP-Mixers, and Vision Transformers to Enhance Image Classification [2.907712261410302]
  We build upon and improve previous work exploring the complementarity between different architectures.<n>We preserve the integrity of each architecture and combine them using ensemble techniques.<n>A direct outcome of this work is the creation of an ensemble of classification networks that surpasses the accuracy of the previous state-of-the-art single classification network on ImageNet.
  arXiv  Detail & Related papers  (2025-04-12T04:32:52Z)
• A Survey of Model Architectures in Information Retrieval [59.61734783818073]
  The period from 2019 to the present has represented one of the biggest paradigm shifts in information retrieval (IR) and natural language processing (NLP)<n>We trace the development from traditional term-based methods to modern neural approaches, particularly highlighting the impact of transformer-based models and subsequent large language models (LLMs)<n>We conclude with a forward-looking discussion of emerging challenges and future directions.
  arXiv  Detail & Related papers  (2025-02-20T18:42:58Z)
• EM-DARTS: Hierarchical Differentiable Architecture Search for Eye Movement Recognition [20.209756662832365]
  Differentiable Neural Architecture Search (DARTS) automates the manual process of architecture design with high search efficiency.<n>We propose EM-DARTS, a hierarchical differentiable architecture search algorithm to automatically design the DL architecture for eye movement recognition.<n>We show that EM-DARTS is capable of producing an optimal architecture that leads to state-of-the-art recognition performance.
  arXiv  Detail & Related papers  (2024-09-22T13:11:08Z)
• Unsupervised Graph Neural Architecture Search with Disentangled Self-supervision [51.88848982611515]
  Unsupervised graph neural architecture search remains unexplored in the literature. We propose a novel Disentangled Self-supervised Graph Neural Architecture Search model. Our model is able to achieve state-of-the-art performance against several baseline methods in an unsupervised manner.
  arXiv  Detail & Related papers  (2024-03-08T05:23:55Z)
• Automated Fusion of Multimodal Electronic Health Records for Better Medical Predictions [48.0590120095748]
  We propose a novel neural architecture search (NAS) framework named AutoFM, which can automatically search for the optimal model architectures for encoding diverse input modalities and fusion strategies. We conduct thorough experiments on real-world multi-modal EHR data and prediction tasks, and the results demonstrate that our framework achieves significant performance improvement over existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-20T15:14:14Z)
• A General Purpose Neural Architecture for Geospatial Systems [142.43454584836812]
  We present a roadmap towards the construction of a general-purpose neural architecture (GPNA) with a geospatial inductive bias. We envision how such a model may facilitate cooperation between members of the community.
  arXiv  Detail & Related papers  (2022-11-04T09:58:57Z)
• Neural Architecture Search based on Cartesian Genetic Programming Coding Method [6.519170476143571]
  We propose an evolutionary approach of NAS based on CGP, called CGPNAS, to solve sentence classification task. The experimental results show that the searched architectures are comparable with the performance of human-designed architectures.
  arXiv  Detail & Related papers  (2021-03-12T09:51:03Z)
• A Semi-Supervised Assessor of Neural Architectures [157.76189339451565]
  We employ an auto-encoder to discover meaningful representations of neural architectures. A graph convolutional neural network is introduced to predict the performance of architectures.
  arXiv  Detail & Related papers  (2020-05-14T09:02:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.