ChatGPT, which is hailed by many as the "dark horse" in the field of language processing, can it be applied to writing scientific research papers? Recently, a team led by Roy Kishony, a biologist and data scientist at the Technion-Israel Institute of Technology, conducted a series of studies on this issue, and the results were published in the journal Nature. Let's take a look at this thought-provoking study.
Experimental history
In the preliminary stages of the experiment, Professor Kishony and graduate student Tal Ifargan developed an automated system that acts like a dedicated tutor, guiding ChatGPT step-by-step through the writing of a research paper. The data they used came from a publicly available dataset from the U.S. Centers for Disease Control and Prevention, which included health data on 250,000 people about their exercise habits, diet, and diabetes status.
Then, the system kicks in, letting ChatGPT start writing the code for data exploration. ChatGPT had errors on its first try, but when the system reported those errors, ChatGPT successfully fixed the code. After that, under the guidance of the system, ChatGPT determined the research goal: to explore how exercise and diet affect the risk of diabetes. Then, according to the data analysis plan proposed by itself, it wrote the analysis code and generated the results.
With these results, the system further guides ChatGPT to write essays . In order to ensure the accurate citation of the literature, they also specially opened the permission to access the literature search engine for ChatGPT. Eventually, ChatGPT completed a research paper.
Experimental results
However, while the ChatGPT paper appears to be perfect on the surface, after a more in-depth review, the research team found some concerns.
For example, ChatGPT claimed in the paper that the study "filled a gap in the literature"—a common claim in scientific papers, but in this particular case, that statement was inaccurate. In fact, what it explores about the relationship between diet, exercise, and diabetes risk does not shed new light on existing medical research.
Professor Kishony also pointed out that this application of ChatGPT may also interfere with the experimental results of the scientific research team. For example, when scientists conduct scientific experiments, they often make multiple hypotheses for a data set, but ChatGPT's data analysis may report only those hypotheses that produce significant results. This behavior may induce scientists to ignore important information in research, thereby affecting the final experimental results.
In addition, it is worth noting that the application of ChatGPT, an AI tool that can automatically generate papers, may lead to academic journals being flooded with a large number of low-quality papers.
Therefore, although ChatGPT has demonstrated the ability to write scientific research papers, AI tools cannot yet replace the research and analysis of scientific researchers, and even unreasonable use may interfere with research results. The application and practice of AI tools in the field of scientific research still needs to be studied and optimized. In particular, how to prevent AI from "fabricating" data or citations, and how to ensure the quality of the papers it produces are urgent problems to be solved.