Skip to content
CodeKruPro

Tests that mirror the bug

30/99

Module · Reviewing AI-written tests

intermediate

about 30 min

What this lesson teaches

The idea

Asked to test a page, an assistant often copies what the page shows into the test. If the page is wrong, the test now protects the bug.

Why a tester cares

Expected values must come from the spec, worked out by hand, and only then be compared with the app.

Words you will meet

Spec:
the written description of how the feature should behave.
Mirror test:
a test whose expected values were copied from the app itself.

What you will do

  1. 1Run the copied test
  2. 2Work out the spec values
  3. 3Fix it and add a rule
  4. 4Submit
Jump to practice

Ask an assistant to "write a test for this page" and it will often look at what the page shows and assert exactly that. If the page is wrong, the test now protects the bug: it passes today and will fail on the day someone fixes it.

The test in the editor was written that way. Its comment even says the values were copied from the screen. Run it: it passes, because the app has the bug it copied.

Your task

  1. Work out from the spec what SAVE10 on two mugs should give.
  2. Rewrite the test so its expected values come from the spec. It must fail on today's buggy app.
  3. Add a second test for another SAVE10 rule from the spec, so the fixed app is checked on more than one rule.

Sign in to practise

The lesson is free to read. A free account lets you talk to the practice bots, run your code and save your score. You will come back to this lesson.

Sign inCreate a free account
ShopLab specShow
ShopLab pricing and checkout rules

Products: Coffee mug (MUG) Rs 299 · Notebook (NOTE) Rs 149 · Wireless headphones (PHONES) Rs 1,999 · Cotton T-shirt (TEE) Rs 499
Each cart line has a quantity from 1 to 10.

Coupons (one per order; applying a new coupon replaces the old one)
  SAVE10   10% off the subtotal, rounded down to the rupee. Needs a subtotal of Rs 500 or more.
           Below that the message is "SAVE10 needs a subtotal of Rs 500 or more." and the discount stays Rs 0.
  FLAT100  Rs 100 off. Needs a subtotal of Rs 999 or more.
  OLD20    Expired. Message: "This coupon has expired."
  Anything else: "Coupon not found."

Shipping: Rs 49 when the subtotal after discount is under Rs 500; free from Rs 500.
Total = subtotal - discount + shipping.

Confirmation page: heading "Order confirmed", the order ID (SL-12345), one line per item such as
"Notebook × 2", and "Total: Rs 1,234". Item lines and the total load a moment after the page.

For tests
  The app's address is in the CK_APP_URL environment variable. Pages: /shop/, /shop/cart, /shop/help.
  POST {CK_APP_URL}/api/cart with {"sku": "MUG", "qty": 2} fills the cart. page.request shares the browser's
  cookies, so a test can set up its cart through the API and then check the page.
  Test IDs (data-testid): message, subtotal, discount, shipping, total, order-id, order-line, order-total.
1

Your code

Write it in
Both are graded the same way.
against today's ShopLab (it has a bug)

How this is checked: When you submit, your tests run against 3 versions of the ShopLab store: 1 correct and 2 with a planted fault. Good tests pass on every correct version and fail on every faulty one. Run tries them on the practice copy first; the result table shows each version.

Core checks earn 50 points and the harder checks the other 50. You can submit as many times as you like.

About this lesson

Tests that copy the bug from the screen: deriving expected values from the spec

AI Testing for QA EngineersIntermediateReviewing AI-written tests

How it works

Ask an assistant to write a test for a page and it often reads what the page shows and asserts exactly that. If the page is wrong, the test now protects the bug: it passes today and fails on the day someone fixes it.

The defence is a habit, not a tool. Expected values come from the specification, worked out by hand, and only then compared with what the app shows. A comment saying values were copied from the screen is a review red flag.

Adding a second test for a different rule in the same feature makes sure the fixed app is checked from more than one angle.

About the assignment

Rewrite a copied test so its expected values come from the spec and it fails on today's buggy app, then add a test for another SAVE10 rule.

What you will do

  1. Run the copied test and see it pass on the buggy app.
  2. Work out the correct numbers for two mugs with SAVE10 from the spec.
  3. Rewrite the assertions with those numbers, and add a test for another SAVE10 rule.

Common mistakes

  • Rounding the wrong way

    The spec says how SAVE10 rounds. Use its rule, not the one you expect.

  • Only fixing the number

    A second rule, such as the minimum subtotal, guards the feature after the fix too.

In a real test suite

Mirror tests are common in suites generated from screenshots or live pages. Teams catch them by asking where each expected value came from during review.

Related lessons

Next lessonMutation testing as reviewAll lessons in the courseAI Testing for QA Engineers