Test planning, estimation and metrics

Planning questions check whether you can think beyond single test cases: what to test, how long it will take, when you are finished and how you show progress. They also include small calculations, such as a three-point estimate or a defect density, that are easy marks if you know the formula. This article covers the planning words first, then estimation, then metrics.

Test strategy and test plan

A test strategy says how testing is done in general: which levels and types, the automation approach, tools, environments and how risk drives the work. It is often written once for a company or a large programme (a group of related projects) and changes rarely.

A test plan applies the strategy to one project or release (one version of the product sent to users). It usually holds the scope (features in and out), the approach, entry and exit criteria, environments and tools, roles, the schedule and estimates, risks and assumptions, and the reports the team will produce. Agile teams often keep it to a short page and update it as they go.

Entry and exit criteria

Entry criteria say what must be true before an activity starts: requirements reviewed, test cases ready, the environment and data available, and a build (the version of the software to test) that passes the smoke tests (quick checks that the build works at all). Exit criteria say what must be true to finish, for example all high-priority tests run, at least 95% passed, no open critical or high defects, and the remaining defects reviewed by the product owner. In Agile teams the definition of ready and the definition of done play these roles (see SDLC, STLC and Agile).

"When do you stop testing?" is a classic question. Testing cannot prove there are no defects, so the honest answer is: when the agreed exit criteria are met, or when time runs out and the stakeholders have seen the remaining risk and accepted it.

The traceability matrix

A requirements traceability matrix (RTM) links each requirement to the tests that cover it, and often to their results and defects.

Requirement	Test cases	Status	Defects
R12 Apply coupon	TC-40, TC-41, TC-42	2 passed, 1 failed	BUG-311 (a bug ID)
R13 Remove coupon	TC-43	Passed	None
R14 Coupon limit per user	None	Not covered	None

The empty row is the point: R14 has no tests, so it is a coverage gap. When R12 changes, the matrix tells you which three cases to update. Forward traceability (requirement to tests) shows every requirement is tested; backward traceability (test to requirement) shows every test has a reason to exist. Most teams now build it from links between stories and test cases in their tools.

Estimation

An estimate always rests on assumptions, so state them, and estimate tasks rather than multiplying a test case count. Common techniques:

• Ratios from past projects. A person-day is one person working for one day. If development and test effort were in the ratio 5 : 2 (5 parts development to 2 parts testing) and development needs 250 person-days, testing needs 250 x 2 / 5 = 100 person-days.
• Extrapolation, which means extending a trend from early data, such as the average test effort of the last three sprints.
• Wideband Delphi: experts estimate alone, discuss the differences and repeat. Planning poker, a card game where each person shows an estimate at the same time, is a version of it.
• Three-point estimation: E = (optimistic + 4 x most likely + pessimistic) / 6. With 4, 6 and 14 days, E = (4 + 24 + 14) / 6 = 7 days. The spread (how far the real value may be from the estimate) is (14 - 4) / 6, about 1.7 days.

"How many test cases can you write in a day?" has no single answer. Say it depends on complexity, give honest numbers from your own work, and explain that you estimate analysis, design, data, execution, retesting, regression and reporting.

Prioritising tests

When there is not enough time, rank tests by risk first: where a failure would hurt most and is most likely, such as payments and login. Then use requirement priority, recent changes and bug history. Respect dependencies: if an important test needs data that a less important one creates, run that one first. Share the list of what will not be tested, so skipping it is a recorded decision.

Metrics

A useful metric helps someone decide something. Teams track progress (tests planned, run, passed, failed, blocked), coverage (how much of the requirements or code the tests reach), and open defects by severity (how badly each one hurts). Two formulas come up often:

• Defect density is defects divided by size. Size is usually counted in lines of code. Module A (one part of the application) with 24 defects in 8,000 lines has 3 per thousand lines; module B with 30 in 15,000 lines has 2. A is riskier even though it has fewer defects.
• Defect detection percentage (DDP, often called defect removal efficiency) is defects found by testing divided by all defects, including those found later. 180 found in testing and 20 found by users gives 180 / 200 = 90%.

How to answer in an interview

Write the formula, put the numbers in, and say what the result means for a decision. For "when do you stop?", talk about exit criteria and accepted risk.

Common mistakes

• Taking a plain average in a three-point estimate.
• Comparing raw defect counts across modules of different sizes.
• Reporting numbers with no sentence about the remaining risk.

Recap

The strategy is general and the plan is per project. Entry criteria start work and exit criteria end it. The RTM shows coverage gaps and change impact. Estimate tasks, state assumptions, and report metrics that lead to decisions.