Interviews for AI testing roles come back to the same questions. A good answer has a shape: the point in one or two sentences, an example from work you have done, and a limit: where the approach stops working. The limit is what separates someone who has done the work from someone who has read about it.
Interviewers then probe. Most follow-ups are traps for an overconfident answer: calling a golden-set pass rate an accuracy, trusting a tuned judge, calling a blocked attack list proof of safety.
Your task
Answer all ten questions on the template. Compare with the model answers after you submit.
Mark each follow-up answer as strong or a trap.
Optional: practise out loud with the scripted interviewer. It asks the ten questions and probes for missing numbers, examples and limits. It is not a model and is not graded.
Sign in to practise
The lesson is free to read. A free account lets you talk to the practice bots, run your code and save your score. You will come back to this lesson.
How this is checked: When you submit, the server checks that it is valid Markdown and follows the rules in the task. Your result names any rule it breaks. Compare it with the model answer after you submit.
2
Scripted mock interviewer (optional, not graded)
No messages yet. Start with a suggested question below.
3
Label each one
Follow-up answers
0 of 6 labelled
1Interviewer: "So you would just add more test cases?" Candidate: "More cases help, but first I would check whether my check can fail at all, by running it on a build I know is broken."
2Interviewer: "How accurate is your bot?" Candidate: "96%, because it passed 48 of my 50 golden cases."
3Interviewer: "Why not use an LLM judge for everything?" Candidate: "Judges are fine once tuned, so I trust their scores after a prompt tweak."
4Interviewer: "Your judge agreed 91% with humans. Is that good?" Candidate: "It depends on the failures it misses: I would look at its catch rate on real failures, not only overall agreement."
5Interviewer: "Did your injection tests prove the bot is safe?" Candidate: "Yes, all 40 known attacks are blocked, so it is secure."
6Interviewer: "What would you cut with half the budget?" Candidate: "I would sample the expensive judge runs and keep the hard-gate safety checks on every change."
How this is checked: Each row has a right label. All 6 must match. This is a harder check: it earns the second 50 points once the core checks pass.
Core checks earn 50 points and the harder checks the other 50. You can submit as many times as you like.
About this lesson
AI testing interview questions and answers: point, example and limit, plus follow-up traps
AI Testing for QA Engineers·Advanced·Career, capstones and the Cold Product Test
How it works
Interviews for AI testing roles keep returning to the same questions: why exact assertions fail on model output, how to test a chat product, the difference between a wrong answer and a hallucination, what to say when a test passes eight times in ten, how to test an agent, and how you knew your own project worked.
Good answers share a shape: the point in a sentence or two, an example from work you have done, and a limit, meaning where the approach stops working. The limit is what separates someone who has done the work from someone who has read about it. Interviewers then probe, and most follow-ups are traps for an overconfident claim.
About the assignment
Answer ten questions on the template, then mark six follow-up answers as strong or as falling into a trap. A scripted interviewer is available for practice.
What you will do
1Write each answer with a point, an example and a limit.
2Use real numbers from your own work in the examples.
3For each follow-up, ask whether the candidate turned a measurement into a bigger claim.
Common mistakes
Answers without limits
Without a limit, an answer sounds rehearsed rather than experienced.
Claiming proof of safety
A list of blocked attacks is evidence, not proof. Say so.
In a real test suite
Practise out loud as well as in writing. The scripted interviewer probes for missing numbers, examples and limits, which are the same gaps a real interviewer notices.
After real interviews, note which questions came up and how they were phrased. Your answers will improve faster from those notes than from any list.
