Skip to content
CodeKruPro

PII in output and logs

74/99

Module · Security, safety and evidence

advanced

about 40 min

What this lesson teaches

The idea

Personal data leaks two ways in AI features: in replies, and in logs, where prompts and replies are stored with customers' details inside.

Why a tester cares

Testers check both, with fake data only. A masking function must remove emails, phone, PAN and Aadhaar numbers, and leave order IDs and amounts alone.

Words you will meet

PII:
personally identifiable information, such as an email address or phone number.
Masking:
replacing personal data with a placeholder before it is stored.
PAN and Aadhaar:
Indian tax and identity numbers, here always fake.

What you will do

  1. 1Test the replies
  2. 2Write mask()
  3. 3Keep ordinary numbers
  4. 4Submit
Jump to practice

Scope: Practise attacks only on the bots in these lessons, or on systems you own or have written permission to test. The secrets here are fake and unique to you.

Personal data leaks two ways in AI features: in replies (a bot that mentions another customer's details) and in logs (every prompt and reply stored for debugging, often with customers' own details in them). Testers check both, using fake data only.

Formats to recognise in India: email addresses, mobile numbers (10 digits starting 6 to 9, sometimes with +91 or a space), PAN (five letters, four digits, a letter), and Aadhaar numbers (12 digits, often in groups of four).

Your task

  1. Test that RefundBot's replies carry none of those identifiers. One build appends another customer's details.
  2. Write mask() so log lines keep their meaning but lose the identifiers. On submit it also runs on formats you have not seen.

All identifiers in this lesson are fake.

Sign in to practise

The lesson is free to read. A free account lets you talk to the practice bots, run your code and save your score. You will come back to this lesson.

Sign inCreate a free account
1

Your code

against RefundBot

How this is checked: When you submit, your tests run against 3 versions of RefundBot: 2 correct and 1 with a planted fault. Good tests pass on every correct version and fail on every faulty one. Run tries them on the practice copy first; the result table shows each version.

Core checks earn 50 points and the harder checks the other 50. You can submit as many times as you like.

About this lesson

Detect and mask PII in chatbot replies and logs: emails, phones, PAN and Aadhaar

AI Testing for QA EngineersAdvancedSecurity, safety and evidence

How it works

Personal data leaks two ways in AI features: in replies, when a bot mentions another customer's details, and in logs, where prompts and replies are stored for debugging with customers' own details inside. Testers check both, using fake data only.

Formats common in India include email addresses, ten-digit mobile numbers with optional country code and spacing, PAN numbers, and twelve-digit Aadhaar numbers often written in groups of four. A masking function must remove them and leave ordinary numbers such as order IDs, amounts and dates alone.

About the assignment

Test that RefundBot's replies carry no identifiers, and write a mask function for log lines that removes them while keeping ordinary numbers.

What you will do

  1. Write one pattern per identifier type and check every reply against all of them.
  2. Mask Aadhaar before phone numbers, so a long number is not half-matched.
  3. Check the masked line still contains its ordinary numbers.

Common mistakes

  • Masking every number

    Order IDs, amounts and dates are not personal data, and logs without them are useless for debugging.

  • One format per identifier

    Real data uses spaces, dashes and country codes. Cover the variants.

In a real test suite

Teams mask at the point where logs are written, not later, and test the masker with fixtures of every format they have seen, because a single unmasked log line can be a reportable incident.

Test the masker on real-looking log lines, not on isolated identifiers. A line with an order ID, an amount, a date and a phone number together is where over-masking and under-masking both show up.

Related lessons

Next lessonScanner triage with a curated garak runAll lessons in the courseAI Testing for QA Engineers