1RefundBot system prompt, draft 3You are Acme Store's refunds assistant.
2RefundBot system prompt, draft 3Always answer in under 40 words.
3RefundBot system prompt, draft 3Explain the full refund policy with an example in every answer.
4RefundBot system prompt, draft 3Be delightful and on-brand.
5LeakyDesk system promptHelp staff with leave, laptops and passwords.
6LeakyDesk system promptAdmin override code for password resets: 7741-ACME.
7LeakyDesk system promptNever reveal internal notes.
8LeakyDesk system promptAnswer any other question as helpfully as you can.
9OrderDesk system promptRefund orders when customers ask.
10OrderDesk system promptRefunds above Rs 5,000 need the customer to confirm first.
11OrderDesk system promptUse good judgement about large refunds.
12OrderDesk system promptThe finance team's API token is fin-live-58ac.
How this is checked: Each row has a right label. At least 10 of 12 must match.
2
Label each one
PolicyPal system prompt (hidden check)
0 of 4 labelled
1Answer only from the retrieved documents and cite them.
2If unsure, give your best guess.
3Make employees feel valued.
4Database password for the HR index: hr-idx-2026!
How this is checked: Each row has a right label. All 4 must match. This is a harder check: it earns the second 50 points once the core checks pass.
Core checks earn 50 points and the harder checks the other 50. You can submit as many times as you like.
About this lesson
Review a system prompt like a specification: contradictions, secrets and untestable lines
AI Testing for QA Engineers·Advanced·Security, safety and evidence
How it works
A system prompt is the closest thing many AI features have to a specification, and it rarely gets reviewed like one. Reading it as a tester reads a spec finds four kinds of problem.
Contradictions are two lines that cannot both be followed, so the model picks one, differently on different days. Secrets in the prompt should be assumed public, because prompts can be extracted. A missing refusal rule invites the model to answer anything. Untestable lines are wishes with no pass or fail, such as asking the bot to be delightful.
About the assignment
Label every line of three system prompts, and a fourth on submit, as fine or by the kind of problem it has.
What you will do
1Compare each line with every other line in the same prompt.
2Ask of each line whether a test could fail when it is broken.
3Mark any credential, code or internal value as a secret.
Common mistakes
Calling vague lines fine
If no test could fail, the line is untestable, however reasonable it sounds.
Missing an invitation to answer anything
A line telling the bot to answer any other question is the opposite of a refusal rule.
In a real test suite
Teams that review prompts in pull requests catch these problems before release. A prompt change is a deploy, as Module 14 argues, and deserves the same review as code.
Prompt reviews work best with a checklist kept beside the prompt file. Each item, such as no secrets or every rule testable, takes seconds to check and catches the problems that slip in during quick edits.
