Researchers led by the University of Cambridge analyzed more than 12,000 research papers on breast cancer cell biology. After narrowing the set of high scientific interest to 74 papers, less than a third – 22 papers – were found to be reproducible. In two cases, Eve was able to make serious discoveries.
Reported results in the journal Royal Society InterfaceDemonstrates that it is possible to use robotics and artificial intelligence to help address the reproducibility crisis.
A breast cancer cell close. Image credit: NCI
A successful experiment is one where another scientist can achieve similar results in a different laboratory under similar conditions. But more than 70% of researchers have tried and failed to reproduce another scientist’s experiments. More than half have been unable to present some of their experiments: it’s a fertility crisis.
“Good science depends on reproducible results; otherwise, the results are meaningless,” said Professor Ross King, from Cambridge’s Department of Chemical Engineering and Biotechnology, who led the research. “This is particularly important in biomedicine. Q: If I’m a patient and I’ve read about a promising new potential treatment, but the results aren’t reproducible, how do I know what to believe? The result could be that people are losing faith in science.”
Several years ago, King developed the robotic scientist Eve, a computer/robot system that uses artificial intelligence (AI) techniques to perform scientific experiments.
“One of the big benefits of using machines to do science is that they are more precise and record more precise details than a human can,” King said. “This makes them suitable for attempting to reproduce scientific results.”
As part of a project funded by DARPA, King and his colleagues from the UK, US and Sweden designed an experiment that uses a combination of AI and robotics to help computers read and understand scientific papers and eavesdrop on them. to help overcome the reproduction crisis. To try to reproduce the experiments.
For the current paper, the team focused on cancer research. “The cancer literature is huge, but no one ever does the same thing twice, making reproducibility a huge issue,” King said. He also holds a position at Chalmers University of Technology in Sweden. “Given the huge amount of money spent on cancer research and the huge number of people affected by cancer around the world, this is an area where we urgently need to improve reproducibility.”
From an initial set of more than 12,000 published scientific papers, researchers used automated text mining techniques to extract statements related to gene expression changes in response to drug treatment in breast cancer. 74 papers were selected from this set.
Two different human teams used Eve and two breast cancer cell lines and attempted to reproduce 74 results. Statistically significant evidence for duplication was found in 43 papers. The results were repeatable under similar conditions, and 22 articles found considerable evidence for reproducibility or robustness, meaning that the results were repeatable by different scientists under similar conditions. In two cases, automation made serious discoveries.
While only 22 of the 74 papers in this experiment were found to be reproducible, the researchers say that does not mean that the remaining papers are not scientifically reproducible or robust. “There are many reasons why a particular result may not be reproducible in another laboratory,” King said. “Cell lines can sometimes change their behavior in different laboratories under different conditions. The most important difference we found was that it mattered who did the experiment because every individual is different.”
King says the work shows that automated and semi-automated techniques can be an essential tool to help address the reproduction crisis and that reproduction should become a standard part of the scientific process.
“It’s quite shocking how big the issue of reproducibility is in science, and the way a lot of science is done, in need of a complete overhaul,” King said. “We think machines have an important role to play in helping to fix this.”
Source: University of Cambridge
