Chat Gtp is a software that generates texts of any kind from translations to essays to scientific abstracts. What are the potential and what are the risks? Moreover,are we able to distinguish texts written by artificial intelligence from those written by human beings? We tried to ask the students of the master’s degree in biomedical science communication at Sapienza.

ChatGPT is a chatbot software that simulates a conversation with a human being. This artificial intelligence technology is based on a large language model (LLM), a machine learning system that can autonomously learn from a huge text dataset and produce content associated with language and writing. ChatGPT answers the questions that are asked and generates written texts, including translations, articles, summaries, and even stories. It is not the only existing chatbot model but it has aroused more interest than the others because it can produce precise texts and in different languages, it is free and easy to use. Precisely because of its accuracy, as well as enthusiasm, 

it has also aroused controversy and led the company to question the potential but also the risks of developing and using this type of technology. In academia, for example, professors have expressed their fears that students might use ChatGPT for cheating. On the other hand,  however, many professors took the opportunity to discuss the fact that our education is too old for our modern society and that this kind of softwares could represent an important resource for students. For example, having students write an essay with artificial intelligence and then asking them to analyze it, finding critical points and modifying it, could help develop critical thinking in young people. In addition to education, in academia researchers and professors are experimenting ChatGPT. Researchers are using software such as ChatGPT to write essays or to improve their papers. For sure the development of this type of softwares could speed up publications by helping researchers to write better texts from a linguistic point of view. On the other hand, it could lead to a deterioration of the contents and to the dissemination of incorrect information. Often, it happens that ChatGPT produces texts with argumentative fallacies or with incorrect scientific information because it is not yet able to discern among the thousands of sources at its disposal. Not only the world of research and education is affected by the advent of chatbots but in general, all fields concerning language and the production of texts, from journalism to writing to translations.

We asked the students to recognize the texts generated by artificial intelligence. They did not recognize 41% of the texts generated by ChatGPT

As journalists and scientists, we have wondered if the texts generated by artificial intelligence are indistinguishable from those written by a human being. For that reason, we decided to make a game with the students of the master’s degree in Biomedical Scientific Communication of Sapienza. First, we generated three documents with ChatGPT on three topics: stem cells, aging, and one health. The questions we asked to ChatGPT were “Explain the topic” and “Produce a scientific abstract of 500 characters”. We also used the ChatGPT feature which allows you to generate multiple answers to the same question. In this way, for each topic, stem cells aging and one health, we have generated 7 texts including 4 scientific abstracts and 3 introductions to the topic. We then handed the documents to the students and, without revealing that they had all been generated through ChatGPT, we asked them to identify those written by artificial intelligence and those written by us.

According to the students, 59% of the texts were written by ChatGPT. This means that students did not recognize 41% of the texts generated by ChatGPT. Almost half of the texts according to them had been written by a human being.

It is inevitable that ChatGPT will become a tool for use by the community, so it is unthinkable to prevent its use today 

So, what is the difference between a text written by a human being and one written by artificial intelligence?

ChatGPT and other large language modeling software can make coherent and fluid texts but sometimes some information such as numerical and statistical data are incorrect. In addition, language used by artificial intelligence is often not suitable for the style of the text to be generated. This particular AI technology is still under development to this day so it is difficult to predict the future impact. What is certain is that it is inevitable that ChatGPT will become a tool for the use of the community and for that reason preventing its use today is unthinkable.

Sofia Gaudioso, biologist and science cumminicator