Parallel and scheduled runs

Running tests in parallel means running several at the same time, on several threads, processes or machines, so the whole suite finishes sooner. (A thread is a separate line of work inside one program; a process is a separate running program.) Slow pipelines get ignored: if results take two hours, developers stop waiting for them. Interviewers ask how you would cut a long run, how you run tests in parallel safely, and how nightly runs fit next to pull-request checks. A pipeline is the list of automatic steps that builds and tests your code. A flaky test is a test that sometimes passes and sometimes fails with no code change. This article covers the levels of parallelism, scheduling, and the trade-offs of automatic reruns.

First: make tests safe to run in parallel

Parallel runs only help when tests are independent:

• Each test or thread has its own browser session. A shared static driver (one browser object used by all tests) makes tests click in each other's windows; Java suites keep one driver per thread, often in a ThreadLocal (a Java object that gives each thread its own copy of a value), and pytest gives each worker its own fixtures (setup code that tests ask for).
• Each test has its own data: unique users, orders and emails, created through the API. Two tests editing the same account fail at random.
• No test needs another test to run first.

Fix these first; otherwise parallel runs turn a slow suite into a flaky one.

Level 1: parallel inside one machine

TestNG runs tests in threads, configured in the suite file:

<suite name="UI" parallel="methods" thread-count="4">

parallel says what is split across threads (methods, classes, tests or instances), and thread-count caps how many threads run at once. JUnit 5 has its own parallel execution settings, and Surefire (the Maven plugin that runs tests) can also start several JVMs (separate Java processes).

pytest uses the pytest-xdist plugin: pytest -n 4 starts four worker processes, and -n auto uses one per CPU (processor core). Each worker is its own pytest session, so a fixture that is meant to run once per run actually runs once per worker.

Level 2: parallel across machines

Sharding splits the suite into parts that run on separate agents or containers at the same time. Example: four balanced shards of a 60-minute suite finish in about 15 minutes plus set-up (an agent is a machine that runs part of the work). Balance shards by duration, not by test count, or the slowest shard decides the total.

In Jenkins, parallel stages run branches at the same time, each on its own agent:

stage('UI tests') {
    failFast false
    parallel {
        stage('Chrome') {
            agent { label 'linux' }
            steps { sh 'mvn -B test -Dbrowser=chrome' }
        }
        stage('Firefox') {
            agent { label 'linux' }
            steps { sh 'mvn -B test -Dbrowser=firefox' }
        }
    }
}

With failFast true, a failing branch stops the others. The failFast false line in the example lets the other branch finish even if one fails. In GitHub Actions, separate jobs run in parallel by default, and a matrix (a list of values, see the GitHub Actions topic) creates one job per combination (browser, shard number, Python version). Browser capacity matters too: a Selenium Grid (a server that starts browsers for tests) with more nodes, or a cloud grid (a grid run by another company), gives more browser sessions at once.

Scheduling: what runs when

Not every test must run on every change. A common split: on each pull request, unit and API tests plus a small UI smoke set (a few quick checks of the most important features) in about ten minutes; after a deploy to a test environment, a broader API and UI regression; and nightly, the full regression suite, sharded, so that the results are ready in the morning. Jenkins schedules with triggers { cron('H 2 * * *') }, where H is a Jenkins symbol that spreads jobs across the hour so they do not all start at once (the Jenkins topic explains cron). GitHub Actions uses on: schedule with cron in UTC, runs on the default branch, and may start late when busy. A nightly run only has value if someone owns its results every morning.

Reruns: helpful and dangerous

Automatic reruns of failed tests (Surefire -Dsurefire.rerunFailingTestsCount=2, the pytest-rerunfailures plugin's --reruns 2, or a TestNG retry analyzer) can keep a pipeline green through a passing glitch. A glitch is a short failure with no clear cause. Reruns also hide real bugs that appear only now and then, such as race conditions in the product (two actions at the same time giving a wrong result), and they add time. Keep them visible: Surefire, for example, reports tests that passed only on a rerun as flaky. Track flaky tests, give each an owner and fix the cause.

How to answer in an interview

For "Our pipeline takes two hours. What would you do?", answer in this order: measure the slowest stages and tests; make tests independent; run them in parallel threads or workers; shard across agents, balanced by duration; split the suite into pull-request, post-deploy and nightly levels; move data checks from UI tests down to API tests; remove fixed sleeps (waiting a set number of seconds); cache dependencies (keep downloaded libraries between runs). Finish with "and report the times before and after".

Common mistakes

• Turning on parallel runs before tests have their own driver and data.
• Sharding by test count instead of duration.
• Silent reruns that hide flaky tests and product race conditions.
• A nightly run nobody looks at.

Recap

Independence comes first: own driver, own data, no order. Then run tests in parallel inside one machine (TestNG threads, pytest-xdist workers), then across machines (shards, Jenkins parallel stages, GitHub matrices). Run fast checks on every pull request and the full suite on a schedule, and keep reruns visible so flaky tests get fixed.