Ace
Short answer: probably yes, as long as the questions are right. Practice testing is one of the strongest findings in learning research, and an early classroom study found students scored higher on a later quiz after a week of AI-written practice questions. But every study so far also found that a share of AI questions are flawed, and the researchers checked them before students saw them. Use AI questions, but only ones tied to your own material and checked against it.
Why practice questions work at all
Before asking whether AI questions help, it is worth being clear about why any practice questions help. The effect is called retrieval practice, or the testing effect: pulling information out of your memory strengthens it more than putting it in again.
Roediger and Karpicke (2006) showed this with ordinary prose passages. Students who took recall tests remembered substantially more after two days and after a week than students who spent the same time rereading, even though the rereaders felt more confident. A meta-analysis by Adesope, Trevisan and Sundararajan (2017) brought together the wider research and concluded that practice tests are more beneficial for learning than restudying and all the other comparison conditions they looked at.
So the real question about AI is not whether testing works. It is whether questions written by a language model are good enough to give you the same benefit, without teaching you anything wrong.
Do AI-written questions help students learn?
The evidence is early and thin, but the first results are positive.
The most direct test so far is a study by An, Liu, Acharya and Hashmi (2025), posted as a preprint on arXiv, which means it has not yet been peer reviewed. In two undergraduate data science courses with about 60 students, lectures in one two-week cycle included AI-written multiple-choice questions after each 10 to 15 minute segment. In the next cycle there were none. On the quiz that followed each cycle, students averaged 89% accuracy after the cycle with AI questions and 73% after the cycle without them.
That is encouraging, but read it carefully. Students were not randomly assigned, the two quizzes covered different topics, and only short-term retention was measured. The authors list these limits themselves. And the questions students saw had been picked and checked by instructors first, which turns out to be the most important detail.
A second study comes from medical education. Zahn and colleagues (2026) built a system at the University of Cincinnati that generated board-style practice questions from five lectures in a first-year haematology course. Faculty reviewed every question before release. Eighty students used the question bank, and greater use trended towards better performance on related exam questions, though the difference was not statistically significant.
How often are AI practice questions right?
This is where the picture gets mixed. Each study reports a different rate, because they used different models, prompts and subjects.
- Zahn and colleagues (2026): of 565 generated questions, 490 (87%) were judged accurate and in line with exam-writing guidelines after faculty review. That still left 75 that were not.
- An and colleagues (2025): to get five good questions per lecture segment, the instructors usually generated at least 15, and about two thirds did not meet their standards. Problems included mistakes in questions and options, made-up content, hints in the question and ambiguous answers.
- Cheung and colleagues (2023): in a blinded comparison, five assessors rated 50 ChatGPT questions and 50 written by university staff from the same medical textbooks. Overall scores were not significantly different, but the AI questions scored lower on relevance to clinical context, and their quality varied more.
Older, less guided studies found higher error rates still. We cover those in our guide to AI quiz generators from PDF, which also has a checklist for choosing one.
Why the checking step matters so much
Notice what every positive result has in common: a person checked the questions before students used them. In the data science preprint, the authors write that instructors must still manually verify and revise the questions. In the Cincinnati study, a faculty course director led the review.
That matters because retrieval practice strengthens what you retrieve. If a question has the wrong answer marked as correct, practising it risks reinforcing the error instead of the fact. A confident, well-written question with the wrong answer can do more harm than no question at all. Our post on whether you can trust AI study answers explains why AI can be confidently wrong and how to spot it.
When you make questions yourself with a chatbot, nobody is doing that review for you. The useful habit is to treat every AI answer as a claim to check against your notes, not a fact.
How to use AI practice questions well
- Give it your material. Questions based on your lecture slides or textbook chapter are easier to check and closer to what your exam will test than questions on a topic in general.
- Answer before you look. The benefit comes from the effort of recall. Commit to an answer, then check.
- Check each answer against the source. If you cannot find the sentence in your notes that supports it, do not trust it.
- Throw out bad questions. Questions that are ambiguous, give away the answer or test something your course never covered are not worth practising.
- Repeat after a gap. The long-term gains in the research come from testing again days later, not from one long session. See our active recall techniques for ways to space it out.
That checking step is the part Ace is built around. It writes questions only from the file you upload, such as a PDF, PowerPoint or Word document, checks each answer against that file and drops any question that fails the check. Under every answer it shows the sentence from your file that backs it, so you can see the source for yourself.
Frequently asked questions
Are AI practice questions as good as ones written by teachers?
Sometimes close. In Cheung and colleagues (2023), blinded assessors gave ChatGPT and human questions similar overall scores, but the AI ones were less relevant to clinical context and more variable in quality. Teacher-written questions and past papers are still the safer core of your revision.
Is the 89% vs 73% result proven?
No. It comes from a single preprint that has not been peer reviewed, with about 60 students, no random assignment and different quiz topics in each cycle. It is a promising early sign, not proof.
Should I use ChatGPT to make practice questions?
It can work if you paste in your own notes, answer before checking, and verify every answer against the source. Our guide to making a practice test with ChatGPT walks through it.
What if an AI question teaches me something wrong?
Correct it straight away and note why it was wrong. Errors you catch and fix are fine; errors you practise without noticing are the real risk, which is why checking against your material matters.
Sources
- Roediger, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249–255.
- Adesope, O. O., Trevisan, D. A., & Sundararajan, N. (2017). Rethinking the use of tests: A meta-analysis of practice testing. Review of Educational Research, 87(3), 659–701.
- An, Y., Liu, J., Acharya, N., & Hashmi, R. (2025). Enhancing student learning with LLM-generated retrieval practice questions: An empirical study in data science courses. arXiv preprint arXiv:2507.05629.
- Zahn, A., Overla, S., Lowrie, D. J., Zhou, C. Y., Santen, S. A., Zheng, W., & Turner, L. (2026). An artificial intelligence-driven platform for practice question generation. Academic Medicine, 101(3), 279–283.
- Cheung, B. H. H., Lau, G. K. K., Wong, G. T. C., Lee, E. Y. P., Kulkarni, D., Seow, C. S., Wong, R., & Co, M. T.-H. (2023). ChatGPT versus human in generating medical graduate exam multiple choice questions: A multinational prospective study. PLOS ONE, 18(8), e0290691.
