Page 71 - 2025S

64 UEC Int’l Mini-Conference No.54

Detecting Modified AI-Generated Academic Abstracts

Andrew Truong , Akira Utsumi 2
1
1. The University of Oklahoma, Oklahoma, United States
2. Artificial Intelligence eXploration Research Center, The University of Electro Communications, Tokyo, Japan

INTRODUCTION o The feature extraction process will analyze text according to the
following framework:
o Advanced paraphrasing tools (QuillBot, Spinbot) can make AI-
generated academic abstracts appear human-written = ! , " , … , #
o Current detection systems fail when AI content is modified through o where () represents the feature vector for text , and
synonym replacement and restructuring 1 ! () represents individual feature extraction functions such as:
o Critical implications for academic integrity, research credibility, o Syntactic complexity measures
and scholarly publishing
o Lexical diversity metrics
o AI-generated abstracts maintain characteristic consistency
patterns in structure and vocabulary that persist even after o Consistency patterns
modification and have lack of depth in terms of content. 2
o Academic Discourse
o Citation Patterns
METHODOLOGY o The classification output will be determined by:
o Used Standardized prompts: "Write a 200-word academic abstract () = (())
about [topic]” with new chat sessions for each prompt to prevent o where M represents the trained model and () provides both binary
memory bias
classification and confidence scoring for the input text .
o Academic domains: Computer science, medicine, psychology,
engineering, etc.
PRELIMINARY FINDINGS
5 Topics 5 Background: The integration of Background: Artificial intelligence (AI)
30 artificial intelligence (AI) algorithms, has emerged as a transformative
Domains per Prompts 4 LLMs particularly deep learning, has technology in (topic) to (result). The
integration of machine learning
catalyzed the development of (topic).
Domain per Topic These tools promise enhanced (result) algorithms with (topic).
Objective: This study synthesizes Objective: This review examines the
existing peer-reviewed evidence on the current applications, clinical
3000 Total AI Generated Abstracts performance, implementation effectiveness, and implementation
challenges, and clinical impact of AI-
challenges of AI-assisted (topic).
assisted (topic).
Methods: (Previous Studies without Methods: We analyzed (Previous Studies
Citations). Inclusion criteria without Citations).
Figure 1: Academic Abstract Generation Protocol Overview comprised clinical evaluations of AI
models in image interpretation for Results: AI-assisted tools demonstrated
computed tomography, MRI, and significant improvements in (Generated
Statistics with no Citations).
o 1,000 Human abstracts: Scholarly abstracts from Google Scholar radiography. (Critical Data) Implementation revealed workflow
searches across 30 academic disciplines Results: Across 72 eligible studies, optimization benefits, including reduced
o From the 3,000 Original AI abstracts, 1,000 are then AI (Generated Statistics with no radiologist fatigue and enhanced
productivity.
Citations). AI models for lung nodule
paraphrased while another 1,000 are manually paraphrased by detection and breast cancer screening
Humans to ensure a balanced data set for training achieved performance comparable to Conclusions: AI-assisted diagnostic
expert radiologists. (Difficulties and tools show substantial promise for
Training Data Distribution potential cause of errors) improving radiological care quality and
efficiency. (Future Improvements)
Conclusion: AI-assisted diagnostic
tools in radiology exhibit significant
potential to improve accuracy and
1000
1000 3000 1000 efficiency. (Future Research
Suggestions)
1000 Open AI ChatGPT Pure Antropic Claude Sonnet 4
o Citation Analysis: 0% of AI-generated abstracts contain citations
Human Generated o Structural Consistency: All AI models follow identical abstract
Original AI templates regardless of topic domain
Human Paraphrased o Content Depth: AI abstracts provide broad, surface-level overviews
QuillBot Paraphrased lacking domain-specific depth and expertise
Figure 2: Data Distribution o Cross-Model Pattern: Healthcare, engineering, and psychology
abstracts demonstrate identical logical structure with only topic-
o Train a two layer feature extraction and classification model 3 specific vocabulary substitution
INPUT CLASSIFICATION REFERENCES
[1] Odri, G-A., & Yoon, D. J. (2023). Detecting generative artificial
intelligence in scientific articles. Orthopaedics & Traumatology: Surgery
FEATURE OUTPUT & Research.
EXTRACTION [2] Chen, Y., et al. (2023). Token Prediction as Implicit Classification to
Identify LLM-Generated Text. EMNLP 2023.
Figure 3: Analysis Framework Pipeline [3] Gifu, D., & Covaci, S-V. (2025). Artificial Intelligence vs. Human:
Decoding Text Authenticity with Transformers. Future Internet.