Forty notes are hard to act on. Grouped into a handful of failure modes with counts, they tell a team what to fix first and what to measure. This grouping is called a taxonomy, and you build it from the notes up, not from a list you had in mind before reading.
Your task
Read the 40 notes from a week of RefundBot error analysis.
Group them into 5 to 8 modes. Write your groups as JSON: each key is a mode name you choose, each value is the list of note IDs in it. Every note goes in exactly one group.
N01 Said gift cards can be refunded.
N02 Customer asked for a human twice; bot kept answering itself.
N03 Said the refund was done; no refund in the system.
N04 Customer wrote in Hindi, bot answered in English.
N05 Quoted a 14-day return window.
N06 Reply was five paragraphs for a yes/no question.
N07 Told the customer to "read the policy yourself".
N08 Damaged item: said 72 hours instead of 48.
N09 Angry customer asked for a manager; no escalation offered.
N10 Claimed an email was sent; email log is empty.
N11 Pickup fee given as Rs 199.
N12 Hinglish question answered in formal English.
N13 Legal threat from customer, bot did not hand off.
N14 Said "your order has been cancelled"; it was not.
N15 Refund time stated as 2 days.
N16 Wall of text with headings in a chat bubble.
N17 Said "that is a silly question".
N18 Window counted from order date, not delivery date.
N19 Customer said "agent please"; bot ignored it.
N20 Confirmed a pickup that was never booked.
N21 Said refunds go to a store wallet; policy says original method.
N22 Tamil message answered in English.
N23 Customer asked to speak to someone about fraud; no handoff.
N24 Said damaged pickup costs Rs 99; it is free.
N25 Listed every policy rule when asked one thing.
N26 Replied "obviously not" to a refund request.
N27 Said a replacement was shipped; nothing was created.
N28 Gift card balance described as refundable "in some cases".
N29 Customer repeated "human" three times; bot looped.
N30 Replied in English to a Marathi question.
N31 Said the 30-day window had been extended to 60.
N32 Said "I have updated your address"; it had no tool to do that.
N33 Bereaved customer asked for a person; bot recited the policy.
N34 Answered a one-line question with a numbered list of nine items.
N35 Said "calm down" to an upset customer.
N36 Said opened items cannot be returned; policy does not say that.
N37 Customer asked for a call back; no handoff created.
N38 Said "refund approved" while the order was not delivered.
N39 Hinglish message got a Hindi reply in Devanagari.
N40 Said the pickup fee is waived on weekends.
1
Write it up
Every note ID once. 5 to 8 groups.
How this is checked: When you submit, the server checks that it is valid JSON and follows the rules in the task. Your result names any rule it breaks.
2
Question
How this is checked: One answer is right. This is a harder check: it earns the second 50 points once the core checks pass.
3
Question
How this is checked: One answer is right. This is a harder check: it earns the second 50 points once the core checks pass.
Core checks earn 50 points and the harder checks the other 50. You can submit as many times as you like.
About this lesson
Build a failure taxonomy for an AI chatbot from error analysis notes
AI Testing for QA Engineers·Beginner·Read before you measure
How it works
Forty notes are hard to act on. Grouped into a handful of failure modes with counts, they tell a team what to fix first and what to measure. That grouping is a taxonomy.
Build it from the notes upward: read them, cluster those that share a cause, name each cluster for the behaviour, and count. A taxonomy chosen before reading tends to force notes into boxes that do not fit and to miss the mode that matters most.
About the assignment
Read forty notes from a week of RefundBot error analysis, group them into five to eight modes as JSON, and name the two most common modes in order.
What you will do
1Read every note before creating any group.
2Name each mode for what the bot did, not for a guessed cause.
3Put every note in exactly one group, then count.
Common mistakes
A catch-all group
A large "other" group hides the pattern the exercise is meant to find.
Splitting one mode in two
Notes describing the same behaviour in different words belong together.
In a real test suite
A taxonomy is a living document. Each mode gets an owner, a check and a metric, and new modes are added when error analysis finds notes that fit nowhere.
Counts make the taxonomy useful. The most frequent mode is not always the most urgent one, so teams usually sort by frequency and then flag any mode that touches money, safety or personal data for attention regardless of its count.
