Page 64 - 2024S

UEC Int’l Mini-Conference No.52 57

Comparing Efficiency in Difficulty Control Between a Fine-Tuned

Question Generation Model and a Few-Shot Learning Approach

1
Luke CHEY and Masaki UTO 2
1 UEC Exchange Study Program (JUSST Program)
2 Department of Computer and Network Engineering
The University of Electro-Communications, Tokyo, Japan

Keywords: Question Generation (QG), Reading Comprehension, Deep Neural Networks, Difficulty-
Controllable, Few-Shot Learning

Abstract
In this study, we explore the comparative effectiveness of a multi-step AI model trained with fine-
tuned data for generating reading comprehension questions of varying difficulty levels against the per-
formance of freely available pre-trained text generation models using few-shot learning approaches. The
research focuses on the capability of these models to generate questions that accurately reflect different
difficulty tiers, thereby influencing the accuracy rate of responses. To generate questions, we developed
custom programs to evaluate various freely available pre-trained text generation models with diverse
prompting methods styles. For empirical evaluation, we generated a substantial dataset of questions
and evaluated them using 400 (4 models × 100 iterations) AI models trained on answering reading
comprehension questions. We measured the accuracy of responses as an indicator of the question diffi-
culty. Our findings suggest significant differences in performance between the fine-tuned model and the
few-shot learning approach, highlighting the advantages and limitations of each method. This research
provides valuable insights into the development of AI-driven educational tools and the optimization of
question generation techniques for enhanced learning outcomes.