An AI chatbot could generate a fraudulent but authentic-looking scientific medical paper—unleashing Pandora's box

2023-07-06
1 min read.
Monitoring for bogus content
An AI chatbot could generate a fraudulent but authentic-looking scientific medical paper—unleashing Pandora's box
Professor opens Pandora's box (credit: writer's DALL-E prompt)

Researchers from Charles University, Czech Republic used ChatGPT-3 to generate a completely fabricated scientific article in the field of neurosurgery. Questions and prompts were refined as ChatGPT generated responses, allowing the quality of the output to be iteratively improved.

The results of this proof-of-concept study were striking—the AI language model successfully produced a fraudulent article that closely resembled a genuine scientific paper in terms of word usage, sentence structure, and overall composition. The article included standard sections such as an abstract, introduction, methods, results, and discussion, as well as tables and other data. The entire process of article creation took just one hour without any special training of the human user.

However, examination expert readers were able to identify semantic inaccuracies and errors particularly in the references—some references were incorrect, while others were non-existent.

Tracking

As AI continues to advance, it becomes crucial for the scientific community to verify the accuracy and authenticity of content generated by these tools and implement mechanisms for detecting and preventing fraud and misconduct, says the researchers. How this could be achieved is less clear.

“We should at least declare the extent to which AI has assisted the writing and analysis of a paper as a starting point,” suggests author Dr. Pedro L. Ballester. Another possible solution proposed by colleagues is making the submission of data sets mandatory.

Citation: Pedro L. Ballester. Open Science and Software Assistance: Commentary on “Artificial Intelligence Can Generate Fraudulent but Authentic-Looking Scientific Medical Articles: Pandora’s Box Has Been Opened." https://www.jmir.org/2023/1/e46924/ doi: 10.2196/46924



Related Articles


Comments on this article

Before posting or replying to a comment, please review it carefully to avoid any errors. Reason: you are not able to edit or delete your comment on Mindplex, because every interaction is tied to our reputation system. Thanks!

Mindplex

Mindplex is an AI company, a decentralized media platform, a global brain experiment, and a community dedicated to the rapidly unfolding future. Our platform empowers our community to share and discuss futurist content while showcasing AI and blockchain tools that enhance the media experience. Join us and shape the future of digital media!

ABOUT US

FAQ

CONTACT

Editors

© 2025 MindPlex. All rights reserved