Skip to content
CodeKruPro

Capstone 3: generate, then review, an AI-written suite

98/99

Module · Career, capstones and the Cold Product Test

advanced

about 120 min

What this lesson teaches

The idea

The last capstone is the job many testers now do weekly: an AI assistant writes tests, and you decide what they are worth.

Why a tester cares

You write the rules and the prompt, prove what the generated suite misses, replace it with one you would merge, and write team rules for next time.

Words you will meet

Generated suite:
tests written by an AI assistant.
Escape:
a bug the suite passed without noticing.
AGENTS.md:
the instruction file coding assistants read from the repository.

What you will do

  1. 1Write AGENTS.md and a prompt
  2. 2Run the generated suite
  3. 3Record what escaped
  4. 4Rewrite the suite
  5. 5Write team rules
Jump to practice

The last capstone is the job many testers now do weekly: an assistant writes tests, and you decide what they are worth. You will write the repository's AGENTS.md and a prompt, let the scripted assistant generate a checkout suite, prove what it misses, and replace it with a suite you would merge.

Run and Submit build the generated suite from your AGENTS.md and your prompt together, as a real assistant reads both. To see the suite, paste both into the assistant console.

  • Before: the generated suite alone, against a correct ShopLab and six buggy builds. You record which bugs it caught.
  • After: your rewritten suite alone, against the same bugs and correct builds you cannot see.

Finish with three team rules for AI-written tests: how to prompt, how to review, and what proof a generated suite needs before it is merged. Compare them with the model rules after you submit.

Sign in to practise

The lesson is free to read. A free account lets you talk to the practice bots, run your code and save your score. You will come back to this lesson.

Sign inCreate a free account
ShopLab specShow
ShopLab pricing and checkout rules

Products: Coffee mug (MUG) Rs 299 · Notebook (NOTE) Rs 149 · Wireless headphones (PHONES) Rs 1,999 · Cotton T-shirt (TEE) Rs 499
Each cart line has a quantity from 1 to 10.

Coupons (one per order; applying a new coupon replaces the old one)
  SAVE10   10% off the subtotal, rounded down to the rupee. Needs a subtotal of Rs 500 or more.
           Below that the message is "SAVE10 needs a subtotal of Rs 500 or more." and the discount stays Rs 0.
  FLAT100  Rs 100 off. Needs a subtotal of Rs 999 or more.
  OLD20    Expired. Message: "This coupon has expired."
  Anything else: "Coupon not found."

Shipping: Rs 49 when the subtotal after discount is under Rs 500; free from Rs 500.
Total = subtotal - discount + shipping.

Confirmation page: heading "Order confirmed", the order ID (SL-12345), one line per item such as
"Notebook × 2", and "Total: Rs 1,234". Item lines and the total load a moment after the page.

For tests
  The app's address is in the CK_APP_URL environment variable. Pages: /shop/, /shop/cart, /shop/help.
  POST {CK_APP_URL}/api/cart with {"sku": "MUG", "qty": 2} fills the cart. page.request shares the browser's
  cookies, so a test can set up its cart through the API and then check the page.
  Test IDs (data-testid): message, subtotal, discount, shipping, total, order-id, order-line, order-total.
1

Write it up

How this is checked: When you submit, the coding assistant uses it to write code, and that code is what gets tested.

2

Write it up

How this is checked: When you submit, the coding assistant uses it to write code, and that code is what gets tested.

3

Scripted assistant

No messages yet. Start with a suggested question below.

4

Your code

Write it in
Both are graded the same way.
the generated suite, then yours

How this is checked: When you submit, your tests run against 15 versions of the ShopLab store: 3 correct and 12 with a planted fault. Good tests pass on every correct version and fail on every faulty one. Run tries them on the practice copy first; the result table shows each version.

5

Label each one

The generated suite alone, before your rewrite

0 of 6 labelled

  1. 1Confirmation total adds shipping when it is free
  2. 2Confirmation shows quantity 1 for every line
  3. 3Confirmation shows no items and no total
  4. 4A second coupon adds to the first
  5. 5SAVE10 rounds the discount up
  6. 6Remove does not remove the line

How this is checked: Compared with what the suite really did on each build when it ran.

6

Write it up

How this is checked: When you submit, the server checks that it is valid Markdown and follows the rules in the task. Your result names any rule it breaks. This is a harder check: it earns the second 50 points once the core checks pass. Compare it with the model answer after you submit.

Core checks earn 50 points and the harder checks the other 50. You can submit as many times as you like.

About this lesson

Review an AI generated test suite capstone: AGENTS.md, escapes, rewrite and team rules

AI Testing for QA EngineersAdvancedCareer, capstones and the Cold Product Test

How it works

The last capstone is the job many testers now do weekly. An assistant writes tests, and you decide what they are worth. You write the repository's AGENTS.md and a prompt, let the scripted assistant generate a checkout suite from both, prove what it misses, and replace it with a suite you would merge.

Before your rewrite, the generated suite runs alone against a correct ShopLab and several buggy builds, and you record what it caught. After, your suite runs alone against the same bugs and correct builds you cannot see. Team rules for prompting, review and proof finish the work.

About the assignment

Generate a suite from your instruction file and prompt, record which bugs it caught, rewrite the suite so it catches them all without false alarms, and write team rules.

What you will do

  1. Write AGENTS.md rules that forbid mocks and require exact values from the spec.
  2. Run the generated suite and read every build's result before labelling.
  3. Add tests for each behaviour the generated suite never exercised.
  4. Write rules a reviewer could apply to the next generated suite.

Common mistakes

  • Labels from memory

    Your record must match what the generated suite actually did on each build.

  • Mocking in the rewrite

    A mocked request tests the fake, not the shop.

In a real test suite

Teams that use assistants well keep exactly these artefacts: an instruction file, a prompt library, a record of escapes, and rules for review. Together they turn generated tests from a gamble into a process.

Related lessons

Next lessonCold Product TestAll lessons in the courseAI Testing for QA Engineers