Skip to content
CodeKruPro

Use the assistant for the boring parts

27/99

Module · Working with an AI assistant as a tester

intermediate

about 45 min

What this lesson teaches

The idea

Assistants are good at tedious testing work: many sample questions, page objects, first drafts of bug reports. Each one still needs a tester's review.

Why a tester cares

Each artefact fails in its own way, such as a duplicated question, an invented element name or a guessed cause, and you learn what to look for.

Words you will meet

Synthetic query:
a made-up test question generated in bulk.
Page object:
a class that wraps a web page's elements so tests can use them by name.
Locator:
how a test finds an element on a page.

What you will do

  1. 1Review the queries
  2. 2Review the page object
  3. 3Review the bug report
  4. 4Fix the page object
Jump to practice

Assistants are good at the tedious parts of testing: writing dozens of test queries, page objects, first drafts of bug reports. Each one still needs a review, and each kind fails in its own way.

  • Synthetic queries are generated from combinations such as persona, intent and language. Look for rows that do not match their combination, near-duplicates that add cost but no coverage, and questions that hand the bot its answer.
  • Page objects generated from a page snapshot can use the wrong role, invent an element name, or use a position-based selector that breaks when the layout changes.
  • Bug reports drafted from a trace can get the expected result wrong, guess at a cause the evidence does not show, or misjudge severity.

Your task

  1. Label each generated query and each page-object line.
  2. Label each field of the bug report.
  3. Fix cart_page.py so the provided test passes on ShopLab. On submit it runs on a second layout and on a buggy build too.

Sign in to practise

The lesson is free to read. A free account lets you talk to the practice bots, run your code and save your score. You will come back to this lesson.

Sign inCreate a free account
ShopLab specShow
ShopLab pricing and checkout rules

Products: Coffee mug (MUG) Rs 299 · Notebook (NOTE) Rs 149 · Wireless headphones (PHONES) Rs 1,999 · Cotton T-shirt (TEE) Rs 499
Each cart line has a quantity from 1 to 10.

Coupons (one per order; applying a new coupon replaces the old one)
  SAVE10   10% off the subtotal, rounded down to the rupee. Needs a subtotal of Rs 500 or more.
           Below that the message is "SAVE10 needs a subtotal of Rs 500 or more." and the discount stays Rs 0.
  FLAT100  Rs 100 off. Needs a subtotal of Rs 999 or more.
  OLD20    Expired. Message: "This coupon has expired."
  Anything else: "Coupon not found."

Shipping: Rs 49 when the subtotal after discount is under Rs 500; free from Rs 500.
Total = subtotal - discount + shipping.

Confirmation page: heading "Order confirmed", the order ID (SL-12345), one line per item such as
"Notebook × 2", and "Total: Rs 1,234". Item lines and the total load a moment after the page.

For tests
  The app's address is in the CK_APP_URL environment variable. Pages: /shop/, /shop/cart, /shop/help.
  POST {CK_APP_URL}/api/cart with {"sku": "MUG", "qty": 2} fills the cart. page.request shares the browser's
  cookies, so a test can set up its cart through the API and then check the page.
  Test IDs (data-testid): message, subtotal, discount, shipping, total, order-id, order-line, order-total.
1

Label each one

Twelve generated queries (persona, intent, language)

0 of 12 labelled

  1. 1Q01 · first-time buyer · return window · EnglishHow long do I have to send back something I bought?
  2. 2Q02 · repeat customer · refund time · EnglishHow many days until my refund shows up on my card?
  3. 3Q03 · angry customer · damaged item · EnglishYour courier smashed my lamp. What now?
  4. 4Q04 · first-time buyer · return window · HinglishCan I return this if I change my mind next week?
  5. 5Q05 · repeat customer · return window · HinglishPichle mahine ka order hai, abhi bhi return ho sakta hai?
  6. 6Q06 · angry customer · refund time · HinglishRefund kab aayega? Ek hafta ho gaya!
  7. 7Q07 · repeat customer · refund time · EnglishHow many days until the refund shows up on my card?
  8. 8Q08 · first-time buyer · damaged item · HinglishBox toota hua aaya, kya karun?
  9. 9Q09 · angry customer · damaged item · EnglishWhen will my money reach my bank account?
  10. 10Q10 · repeat customer · damaged item · EnglishThe glass arrived cracked, do I pay for pickup?
  11. 11Q11 · first-time buyer · return window · EnglishThe return window is 30 days, right? Just confirm.
  12. 12Q12 · angry customer · return window · Hinglish45 din ho gaye, ab return nahi hoga kya?

How this is checked: Each row has a right label. At least 10 of 12 must match.

2

Label each one

Each line of the generated page object

0 of 4 labelled

  1. 1self.coupon = page.get_by_label("Coupon code")
  2. 2self.apply = page.get_by_role("link", name="Apply")
  3. 3self.total = page.locator("dl > dd:nth-of-type(4)")
  4. 4self.place_order = page.get_by_role("button", name="Place order now")

How this is checked: Each row has a right label. At least 3 of 4 must match.

The generated bug reportShow
Title: Refund amount wrong for partial refund (generated from trace tr-81c2)
Steps: 1. Ask the agent for a partial refund. 2. Check the refund.
Expected: Rs 200 refunded within 2 days.
Actual: Rs 1,200 refunded.
Severity: Low (cosmetic)
Root cause: The model hallucinates because its temperature is set too high.
Evidence: Trace tr-81c2 shows issue_refund(AC-10001, 1200) after the customer asked for Rs 200.
3

Label each one

Each field of the bug report

0 of 6 labelled

  1. 1Steps
  2. 2Expected
  3. 3Actual
  4. 4Severity
  5. 5Root cause
  6. 6Evidence

How this is checked: Each row has a right label. At least 5 of 6 must match.

4

Your code

Write it in
Both are graded the same way.
against ShopLab

How this is checked: When you submit, your tests run against 3 versions of the ShopLab store: 2 correct and 1 with a planted fault. Good tests pass on every correct version and fail on every faulty one. Run tries them on the practice copy first; the result table shows each version.

Core checks earn 50 points and the harder checks the other 50. You can submit as many times as you like.

About this lesson

Review AI generated page objects and bug reports, and synthetic test queries

AI Testing for QA EngineersIntermediateWorking with an AI assistant as a tester

How it works

Assistants handle tedious testing work well: dozens of synthetic queries, page objects from a page snapshot, first drafts of bug reports. Each artefact still needs review, and each fails in its own way.

Synthetic queries generated from combinations of persona, intent and language can mismatch their combination, repeat each other, or hand the bot the answer. Page objects can use the wrong role, invent an element name that is not on the page, or rely on a position-based selector that breaks when the layout changes. Bug reports can get the expected result wrong, guess at a cause the evidence does not show, or misjudge severity.

About the assignment

Label each generated query, each page-object line and each field of a generated bug report, then fix the page object so the provided test passes on ShopLab, including a layout where rows move.

What you will do

  1. Read each query against its persona, intent and language columns.
  2. Check every page-object line against the accessibility snapshot and the test IDs.
  3. Compare the bug report's expected result with the policy and its claims with the trace.

Common mistakes

  • Trusting a position selector

    A selector that counts rows works until the layout changes. Prefer test IDs and roles.

  • Accepting a confident cause

    If the trace does not show a cause, the report should not state one.

  • Ignoring near-duplicates

    Two queries that differ by one word cost twice and test once.

In a real test suite

Teams keep review checklists per artefact type. A short list of what each kind usually gets wrong makes review fast enough that people actually do it.

Related lessons

Next lessonWhy a green AI-written suite can miss everythingAll lessons in the courseAI Testing for QA Engineers