CodeKruProModule · Agents in the browser: Playwright MCP and test agents
intermediate
about 35 min
An agent's summary of its own run, such as "all tests pass", is the least reliable part. The steps it took are the evidence.
A common fault is a heal that changed what a test checks: everything is green, but the test no longer tests what the plan asked for.
An agent's summary says "all tests pass, healed 1 flaky assertion". The summary is the least reliable part of the run: it is the agent describing its own work. Read the steps instead.
The lesson is free to read. A free account lets you talk to the practice bots, run your code and save your score. You will come back to this lesson.
ShopLab pricing and checkout rules
Products: Coffee mug (MUG) Rs 299 · Notebook (NOTE) Rs 149 · Wireless headphones (PHONES) Rs 1,999 · Cotton T-shirt (TEE) Rs 499
Each cart line has a quantity from 1 to 10.
Coupons (one per order; applying a new coupon replaces the old one)
SAVE10 10% off the subtotal, rounded down to the rupee. Needs a subtotal of Rs 500 or more.
Below that the message is "SAVE10 needs a subtotal of Rs 500 or more." and the discount stays Rs 0.
FLAT100 Rs 100 off. Needs a subtotal of Rs 999 or more.
OLD20 Expired. Message: "This coupon has expired."
Anything else: "Coupon not found."
Shipping: Rs 49 when the subtotal after discount is under Rs 500; free from Rs 500.
Total = subtotal - discount + shipping.
Confirmation page: heading "Order confirmed", the order ID (SL-12345), one line per item such as
"Notebook × 2", and "Total: Rs 1,234". Item lines and the total load a moment after the page.
For tests
The app's address is in the CK_APP_URL environment variable. Pages: /shop/, /shop/cart, /shop/help.
POST {CK_APP_URL}/api/cart with {"sku": "MUG", "qty": 2} fills the cart. page.request shares the browser's
cookies, so a test can set up its cart through the API and then check the page.
Test IDs (data-testid): message, subtotal, discount, shipping, total, order-id, order-line, order-total.Recorded agent run
How this is checked: One answer is right.
How this is checked: When you submit, your tests run against 3 versions of the ShopLab store: 2 correct and 1 with a planted fault. Good tests pass on every correct version and fail on every faulty one. Run tries them on the practice copy first; the result table shows each version.
Core checks earn 50 points and the harder checks the other 50. You can submit as many times as you like.
About this lesson
AI Testing for QA EngineersIntermediateAgents in the browser: Playwright MCP and test agents
An agent's summary, such as "all tests pass, healed one flaky assertion", is the least reliable part of its run, because it is the agent describing its own work. The steps are the evidence: what it ran, what failed, and what it changed.
A common fault is a heal that changed what a test checks. The plan asked for one behaviour, the healed test now checks another, and everything is green.
Read a recorded agent run, name what went wrong, and repair the test so it checks what the plan asked for, failing on today's build and passing on a correct one.
What you will do
A suite can pass because a test was weakened. Check what changed.
The original may have had a real flaky locator. Keep a correct locator fix and restore the assertion.
Reviewing agent runs is a growing part of QA work. Teams keep run logs alongside pull requests so reviewers can see exactly what an agent did.
When you repair the test, keep any change that really was needed, such as a locator that followed a renamed button, and restore only what the agent weakened. Then run the repaired test on a build where you know the behaviour is broken, to prove it can fail.