Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers

Large language models such as ChatGPT can produce increasingly realistic text, with unknown information on the accuracy and integrity of using these models in scientific writing. Depending on publisher-specific guidelines, AI output detectors may serve as an editorial tool to help maintain scientific standards.

⚡ This is an original paraphrased summary — not copied from the abstract. Full paper available at the source link below.

• 1 We gathered fifth research abstracts from five high-impact factor medical journals and asked ChatGPT to generate research abstracts based on their titles and journals.
• 2 Most generated abstracts were detected using an AI output detector, 'GPT-2 Output Detector', with % 'fake' scores (higher meaning more likely to be generated) of median [interquartile range] of 99.98% 'fake' [12.73%, 99.98%] compared with median 0.02% [IQR 0.02%, 0.09%] for the original abstracts.
• 3 The AUROC of the AI output detector was 0.94.

This research advances how AI systems learn, reason, and solve problems — with direct implications for automation and scientific discovery.

This summary is based on publicly available metadata and abstract. For the full research paper, visit the original source: