Why your Playwright tests become flaky (and how to stop them)

You start CI: green. You rerun it with nothing changed: red. You rerun it again: green. The whole team knows that test “known to be flaky” — and nobody fixes it because “we do not have time”.

This is not bad luck. It is structural. Almost always, flakiness comes from a small number of repeated patterns. Naming them is already half the work of stopping them.

Here are the 6 causes I see most often in an audit — and how to detect them automatically before they block your next release.

1. waitForTimeout — the flake factory

await page.waitForTimeout(3000);
await expect(page.locator('.success')).toBeVisible();

You assume 3 seconds is enough. Sometimes it is. Sometimes the API takes 3.5 seconds. Sometimes 800 ms would do, and you waste time. Sometimes CI is under load and 3 seconds is not enough.

Playwright auto-retries assertions. It does not retry waitForTimeout. This is the number-one cause of tests that are structurally flaky.

Fix: replace it with an assertion on UI state:

await expect(page.getByRole('alert')).toBeVisible({ timeout: 10_000 });

Doctests flags this pattern in a static scan (waitForTimeout in the specs) — often classified as RACE in audit reports.

2. Bad locators — CSS, nth-child, unstable text

await page.locator('div > span:nth-child(3)').click();
await page.locator('.btn-primary').click();

These selectors break on every CSS refactor, every translation, every A/B test. Playwright retries — sometimes the DOM is “almost” right, sometimes it is not. Flakiness is guaranteed over time.

Fix:

• getByRole('button', { name: /confirm/i })
• getByTestId('checkout-submit')
• a page object model, so changes stay in one place

In an audit, Doctests flags fragile CSS locators, unjustified xpath, and POM bypasses (an inline locator in 40 specs instead of a page object).

3. Animations and transitions you do not wait for

A button becomes clickable after a 400 ms animation. Your test clicks at 200 ms. Sometimes it passes (the animation is skipped in headless CI). Sometimes it does not.

Other cases:

• an overlay or modal that fades in
• a skeleton loader that hides the real content
• virtual scroll that moves the target element

Fix:

• await expect(locator).toBeVisible() before the interaction
• await expect(locator).toBeEnabled() when that matters
• disable animations in the test environment only if you control the front end (a feature flag), not as a blind workaround

The Doctests scan detects interactions with no prior assertion on the element (a direct click after navigation, with no explicit wait on state).

4. Slow APIs, or APIs that are not mocked

Your test fills a form, clicks “Submit”, and asserts success — but the API takes 8 seconds every other day (load, a cold start, a rate limit).

Without a network mock or an explicit timeout on the final assertion, you are playing roulette.

Fix:

• page.route() to stabilise responses in E2E
• expect(..., { timeout: 15_000 }) on the post-API state
• an @slow tag, so those specs stay out of the pull-request smoke

Doctests looks at implicit wait patterns and at specs with no data fixtures and no documented network isolation.

5. Unstable data — the same user, the same cart

test('adds a product', async ({ page }) => {
  await login(page, 'test@example.com');
  // ...
});

Two parallel workers, the same account, the same cart, the same order #42 — a collision. Test A fails because test B emptied the stock.

Other traps:

• a hardcoded date (2024-01-15)
• auto-incremented ids assumed to be fixed
• state shared between tests (no cleanup)

Fix:

• a factory per test (createUser(), createOrder())
• Playwright fixtures with test scope
• a reset test database, or rollback transactions

The scan reports missing factories, repeated hardcoded emails, and specs with no isolation (test.describe.configure({ mode: 'serial' }) used as a bandage).

6. Parallelism — when fullyParallel: true exposes the debt

Playwright parallelises by default. That is a feature — unless your suite assumes an implicit order or global state.

Signals:

• test.describe.serial everywhere (you are hiding the problem)
• files that pass alone locally and flake in CI (4 workers)
• a shared storageState overwritten between tests

Fix:

• isolate data and auth per worker
• tags that separate smoke (safe to parallelise) from regression (serial only if you cannot avoid it)
• workers: 1 temporarily, while you fix it — not as a permanent solution

On the in-depth flakiness audit, Doctests ingests your CI runs and computes the flake rate per test — often correlated with specs that fail only in parallel.

How Doctests detects these problems automatically

I do not guess these patterns by hand across 200 specs. Doctests, my internal engine, scans the repo and produces an actionable report.

Free scan (2 minutes)

On /services/devis: you paste the GitHub URL and get a score from 0 to 100, dimensions (locators, waits, POM structure, CI), and anti-pattern signals. An immediate preview, with no commitment.

Audit Flash, €490 (48 hours)

A deeper code-only scan of your repo:

• 20+ static rules (waitForTimeout, CSS locators, POM bypass, missing fixtures, and more)
• a maturity score and the top findings, prioritised
• an 8–12 page PDF and a remediation plan
• a sample deliverable: /services/exemple-audit-flash

No change to your code. Diagnosis only.

In-depth flakiness audit

Ingest of your latest CI runs (JUnit, JSON report):

• flake rate per test over 30 days
• correlation of flakes with anti-patterns in the spec or the page object
• priority: impact = flake_rate × frequency × pattern severity

You know which 5 tests to fix first — not “the whole suite is flaky”.

What the scan actually reports

Problem	Doctests signal
waitForTimeout	RACE rule — count and a code excerpt
Fragile CSS locators	LOCATOR rule — file and line
POM bypass	Inline interaction outside page objects
Shared data	Duplicate emails or ids, no factory
Opaque CI	No documented retry, artefacts, or shards
Runtime flakes	CI correlation (in-depth audit)

This is not a generic linter. These are rules calibrated for Playwright and TypeScript, enriched by what I see on real engagements.

Where to start this week

1. Run the free scan — you will see whether waitForTimeout and CSS locators dominate
2. Note the top 3 findings — those are your quick wins
3. If CI is red several times a week, use an Audit Flash for a prioritised PDF plan
4. If you have 100+ specs and CI runs, use the in-depth audit for the correlated flaky list

You do not need to rewrite 200 tests. You need to kill the 20% that cause 80% of the noise.

Conclusion

Playwright tests do not become flaky “on their own”. They inherit arbitrary waits, fragile locators, shared data, and uncontrolled APIs — amplified by parallelism.

Doctests exists to make those causes visible in minutes, not after weeks of debug. A scan, a score, the top findings, a plan — then you choose whether you fix it internally or continue with a stabilisation sprint.

The block at the bottom of the page shows how to book an Audit Flash or start a diagnosis of your repo.