Groups, fixtures, listeners and retries

A small pytest suite needs only test functions and fixtures. A large UI suite also needs a way to run only some tests, a way to feed data, code that reacts when a test fails, and a way to rerun failures. pytest covers these with markers, parametrize, hooks and a few plugins. Interviewers ask about each one, often by comparing it with TestNG (a widely used Java test runner), so it helps to know both the pytest name and the idea behind it. If you only use pytest, you can skip the TestNG comparisons.

Markers: grouping and selecting tests

A marker is a label you put on a test with a decorator (a line that starts with @ and sits above a function to add information or behaviour to it):

import pytest

@pytest.mark.smoke
def test_login(driver): ...

@pytest.mark.regression
@pytest.mark.slow
def test_full_checkout(driver): ...

Run one group with pytest -m smoke, or combine markers: pytest -m "regression and not slow". The -k option selects by name instead (test names, class names and marker names): pytest -k "login and not admin". Register your markers so a typo becomes an error (the --strict-markers option does that):

# pytest.ini
[pytest]
testpaths = tests
addopts = -ra --strict-markers
markers =
    smoke: quick checks run on every code change
    regression: the full suite
    slow: takes more than a minute

A marker on a class applies to all its tests, and pytestmark = pytest.mark.smoke at the top of a file marks the whole file.

skip, skipif and xfail

• @pytest.mark.skip(reason="..."): the test never runs.
• @pytest.mark.skipif(condition, reason="..."): the test is skipped when the condition is true, for example on one browser.
• @pytest.mark.xfail(reason="BUG-123"): the test runs but is expected to fail. If it passes, pytest reports it as XPASS ("passed when it was expected to fail"), or fails the run with strict=True. This suits known open bugs, because you notice when they are fixed.

Always give a reason, ideally a ticket id (the number of the bug in your bug tracker).

Data-driven tests

@pytest.mark.parametrize turns rows of data into separate tests with ids such as test_login[admin-True]. Load the rows from CSV or JSON in a helper, add ids= so reports read well, and mark a single row with pytest.param(..., marks=pytest.mark.xfail). Fixtures can take parameters too: @pytest.fixture(params=["chrome", "firefox"]) with request.param runs every test that uses the fixture once per browser.

Sharing data between tests

Use fixtures, not global variables. A scope="session" fixture that logs in once and returns a token serves every test that asks for it. Data that a test changes, such as a cart, stays at function scope so tests remain independent. pytest has no built-in test dependencies, which pushes you towards tests that set up their own state.

Hooks: pytest's listeners

TestNG has listeners. pytest has hook functions: functions with fixed names that pytest calls at fixed points in the run. You write them in conftest.py:

• pytest_addoption adds options such as --browser and --env.
• pytest_configure runs once at the start, for example to register markers.
• pytest_runtest_makereport sees each test's result for every phase. Together with the driver fixture, it is how you take screenshots on failure.
• pytest_collection_modifyitems adds markers to collected tests or reorders them.
• pytest_sessionfinish acts at the end of the run, such as posting a summary.

Plugins such as pytest-html and allure-pytest are built on the same hooks. Much of what needs a listener in TestNG is simply a fixture in pytest, because a fixture wraps each test with yield.

Rerunning failures

pytest remembers failures in its cache folder. pytest --lf reruns only the last run's failures, and pytest --ff runs them first and then the rest. The pytest-rerunfailures plugin (an add-on you install separately) retries a failing test straight away:

pytest --reruns 2 --reruns-delay 1

@pytest.mark.flaky(reruns=3)
def test_payment(driver): ...

The report shows the reruns, and the last attempt decides. To hunt a flaky test, the pytest-repeat plugin runs it many times: pytest --count=50 -x tests/test_cart.py::test_add.

Running a subset

One test: pytest tests/test_login.py::test_valid. One method in a class: pytest tests/test_login.py::TestLogin::test_valid. A planned subset: a marker and -m. Each CI job picks its subset with options, so nobody edits tests to change what runs.

How to answer in an interview

When an interviewer uses TestNG words, map them to pytest: "Groups are markers selected with -m. DataProviders are parametrize. Listeners are hooks in conftest.py, though fixtures do most of that work. Retries come from pytest-rerunfailures, and --lf reruns last time's failures." Then give one short example from your own suite.

Common mistakes

• Using -k smoke when you mean -m smoke. -k matches names, so it also picks up a test called test_smoke_alarm.
• Leaving markers unregistered. A misspelled marker then only gives a warning, and that test drops out of -m runs.
• Using skip for a known bug, so nobody notices when it is fixed.
• Turning on reruns for the whole suite and never looking at which tests needed them.

Recap

Markers group tests and -m selects them. skip, skipif and xfail handle tests that should not run or are expected to fail. parametrize feeds data, session fixtures share expensive values, hooks in conftest.py react to the run, and --lf or pytest-rerunfailures repeat failures.