Papers › When Do Biological Reasoning Models Use Their Biological Inputs?

When Do Biological Reasoning Models Use Their Biological Inputs?

1 Oct 2026arXiv:2610.00898added by Syntology

Ada Fang, Nikitha Thoduguli, Lukas Fesser, Hanlin Zhang, Sham M. Kakade, Marinka Zitnik

Title, abstract, authors and date from arXiv's metadata (CC0); this paper is not in the Papers with Code archive (frozen 2025-07-28).

Biological reasoning models use post-training to connect LLMs to biological foundation model representations and biological text. Their benchmark accuracy is taken as evidence that LLMs reason over these inputs. We test this assumption in six biological reasoning models across DNA, protein, and single-cell tasks. We perturb one biological input while holding the query and other inputs fixed, construct evidence conflicts that pair the foundation model representation of one genome, protein, or cell with the text of another, fit linear probes to the representations the language model receives, and analyze reasoning traces against the biological inputs. Evo2 and ESM3 contribute little to BioReason and BioReason-Pro performance on the evaluated tasks. Shuffling the DNA sequence barely changes BioReason disease prediction accuracy, and in evidence conflicts the two models follow the text in 97.9% and 99.7% of cases. Linear probes trained on the Evo2 and ESM3 representations predict the task targets, so these foundation models encode information relevant to the task, but provide limited overall performance improvement to BioReason and BioReason-Pro. In contrast, foundation model inputs contribute to ChatNT, Prot2Text-V2, and CellWhisperer performance, and differentially expressed genes in the gene sentence contribute to Cell2Sentence-Scale performance. Across SFT and RL checkpoints of BioReason-Pro and 42 BioReason checkpoints, increases in accuracy do not imply greater performance contributions from biological inputs. BioReason traces misstate nucleotide changes, while BioReason-Pro traces describe functions omitted from final predictions under evidence conflicts. We find that current post-training strategies do not ensure that foundation model representations contribute to task performance.

PaperPDF

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

For agents, on Syntology's MCP service (how to connect):

Code

mims-harvard/bio-posttrain found in paper text by SyntologySyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to How Post-Training Shapes Biological Reasoning Models (arXiv:2606.16517; that paper's own run record: 12 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 11 where Syntology's instrument failed) · 4 unverified); nothing checks that they implement this paper's method. report
bowang-lab/BioReason found in paper text by SyntologyApache-2.0Syntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to BioReason: Incentivizing Multimodal Biological Reasoning within a DNA-LLM Model (arXiv:2505.23579; that paper's own run record: 8 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report
bowang-lab/BioReason-Pro found in paper text by SyntologySyntology: not harvested report
instadeepai/nucleotide-transformer found in paper text by SyntologySyntology: not harvested report
ColinFX/Prot2Text-V2 found in paper text by SyntologySyntology: not harvested report
vandijklab/cell2sentence found in paper text by SyntologySyntology: not harvested report
epigen/cellwhisperer found in paper text by SyntologySyntology: not harvested report
mims-harvard/bio-mirage found in paper text by SyntologySyntology: not harvested report

Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Syntology holds the repository link but has not harvested or run code from it.

Results from the paper

The Papers with Code archive ends with its 2025-07-28 snapshot. This paper's arXiv identifier, 2610.00898, was issued in October 2026, after that date, so the archive has no leaderboard rows for it.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections