How is AI changing the way research is done, checked and understood? The FORRT AI in Metascience Online Conference spent three days on that question, with 35+ talks and workshops from people who are studying it or building with it. Most talks are now on the
FORRT YouTube channel, one playlist per day.
Here are the sessions and the talks in each, in the order they ran. Click any talk to read its abstract, and use Watch to open its recording. All times are CEST (UTC+2).
From what frontier models can extract, through a head-to-head test of the tools built on them, to the safeguards that would make an AI-assisted review checkable.
What it takes to make a paper checkable in the first place, what human review actually does, what a computational novelty signal might add, and a tool that checks a paper against what its authors said they would do.
What researchers do with these tools, what open infrastructure and open data would require, who gains or loses when the tools arrive, the rules of use that follow, and what it would take to make research outputs readable by machines in the first place.
09:30Symposium: Can AI Help Us Understand How Findings Move from Animal Studies to Human Studies?
Four talks, mostly by early-career researchers, on tracing the biomedical evidence trail across publications, trial registries and regulatory documents. Chaired by Benjamin Ineichen.
Replication and reproduction as work actually done, with agents in the loop and with the paperwork that makes a result reusable, followed out to whether anyone downstream notices.
13:20Getting our hands dirty: experimenting with agentic reproductions and robustness checks
Three coding agents get the same prompt on two well-known cases: reproduce the result, then test how robust it is. While they run, we look at how coding agents work and how to set up tasks so that their output can be checked. Then we go through what came back. No prior experience needed.
14:40Predicting replicability
Three talks on forecasting which findings will hold, and what such forecasts are for.
16:05Machines that build and test the replication record
First the benchmarks that measure what these agents can actually do, then the two databases being built on that basis with humans kept in the loop, and last the question the day has been circling: can an agent run the checks at human quality?
The other way a result fails to hold. The study is not repeated badly; a defensible alternative analysis simply lands somewhere else. And the alternatives increasingly consult the same model.
The talks whose speakers agreed to publication are on the
FORRT YouTube channel, with one playlist per day:
Monday,
Tuesday and
Wednesday. Talks without a Watch link in the programme were not recorded or are not public. For two of the Wednesday workshops, the playlist has a shorter version.
Code of conduct
The conference followed the
FORRT code of conduct. To raise a concern about anything that happened at the conference, email the organiser at
l.wallrich@bbk.ac.uk. If you would rather not come to us, report it to the FORRT ethics committee, either by contacting any of its members or through the
anonymous reporting form. FORRT’s
confidential advisor is there if you would like to talk something through first.
Questions
Email the organiser, Lukas Wallrich, at
l.wallrich@bbk.ac.uk, or message him on the FORRT Slack.
