Self-healing Playwright scripts — what they actually do (and don't)

"Self-healing tests" gets thrown around as a buzzword, usually by tools that don't actually do it. Let's pin down what it means concretely, and why it matters for any team running Playwright at scale.

The problem self-healing solves

Every QA engineer knows the failure mode:

Test passed yesterday. Today it's red. Reason: the developer changed <button class="btn-primary"> to <button class="cta-button">.

The test logic is correct. The selector is just stale. Multiply this by 200 tests and 5 selector changes per sprint, and 30% of your "failures" are no-ops you fix by editing the selector. That work is what self-healing automates away.

What self-healing actually does

When a test fails, the runner doesn't just give up. It runs an extra step:

Capture state: take a DOM snapshot + screenshot at the failure point
Diagnose: ask "did this fail because the selector matched nothing, or because the assertion was wrong?"
If selector-related: search the new DOM for the most likely replacement element (by role, by adjacent text, by visual position)
Patch: rewrite the test's selector and re-run
Verify: if the patched test now passes, surface the diff for human review

Notice what's not in that list: AI hallucinating new test logic, AI inventing assertions, AI deciding "this is fine, mark it green." Self-healing is bounded — it only changes selectors, not intent.

The under-the-hood loop

In SpacePilot QA's implementation, every test failure runs:

1. Re-fetch the failing page (authenticated session if needed)
2. Diff: what changed since the script was generated?
3. Ask the LLM: "which element in the new DOM matches the intent of the
   old selector?" — passing both the old line and the new DOM as context
4. Apply the patch
5. Run only the patched test (not the full suite — fast feedback)

This is much narrower than "AI rewrites my test." The LLM has one job: re-anchor one selector at a time, given full context of what it used to point to.

Why bounded healing is the point

Unbounded "AI fixes my tests" sounds nice in marketing but breaks in practice. If the assertion was wrong (the developer changed product behavior, not just markup), you want a red test — not an AI-rewritten one that papers over a bug.

Bounded self-healing draws the line carefully:

✅ Selector changed → heal it
✅ Element moved in the DOM → re-anchor
✅ Class renamed → swap it
❌ New required field added → fail loudly (this is real)
❌ Endpoint moved → fail loudly (this is real)
❌ Validation error message text changed → fail loudly (intentional or bug, your call)

A self-healing tool that doesn't fail loud on real regressions is worse than a non-healing one. The whole point of automation is to catch breakage.

What you save

Realistic numbers from our usage:

30-40% of failures are pure selector drift → fully auto-healed
20% are flaky (timing, network) → re-run instead of heal, same effect
40-50% are real regressions → reach the human, who's no longer drowning in noise

The biggest win isn't speed. It's that your QA team stops being a triage operation. When a red build hits Slack, it actually means something.

When self-healing doesn't help

Be honest about the limits:

Major refactors (whole pages rewritten): healing has too much to chew on, treat the test as rewrite-fodder
New flows you haven't authored: nothing to heal from
Semantic changes (the button now does X instead of Y): heal would only mask a real bug

If you're shipping a redesign, expect to regenerate test cases. Self-healing is for the steady-state, not for migrations.

The honest takeaway

Self-healing tests are useful, real, and overhyped. They eliminate maintenance work for a specific category of failure — selector drift — and that category is bigger than most teams admit. They don't replace test design, don't auto-detect regressions you didn't write tests for, and don't make a bad suite good.

But if you've spent any time renaming .btn-primary references at 11pm on a Friday, you'll feel the difference.