What we won’t take on

The first question on a migration brief should not be how to convert the tests. It should be which ones to delete.

Delete first, migrate second

Most Selenium suites contain a layer of tests that have stopped earning their keep: assertions that check element presence on a page that redesigns every sprint; screenshot comparisons that have drifted so far from the baseline that failure has become the normal state; tests written to demonstrate that the framework could run, not to protect a behaviour anyone still cares about.

Converting these is not migration. It is debt relocation.

The criterion we apply is straightforward: could this test catch a regression that matters to the business? If the answer requires more than one sentence to justify, that is a signal worth acting on. Applied strictly, this criterion removes a large share of the tests before a single line of Playwright code is written.

Triage of this kind is the most valuable work in a migration. We will do it, and we will tell you what we find, including when the answer is that the suite should be rebuilt from a smaller, better-understood core rather than converted in full.

When the framework is not the problem

Selenium’s architecture is slow and the WebDriver protocol is painful. But the flakiness in most large suites is not a Selenium problem at its root. If the application has no reliable way to reach a known state (no seedable database, no fixture API, no mechanism to create a user and set up the data it needs) then Playwright will surface the same failures, just faster.

Vrbo/Expedia’s public engineering write-up documented a Selenium suite approaching a 50% pass rate, with developers routinely re-running the build to obtain a clean result. The problem was not WebDriver. It was that the application had grown beyond what the test infrastructure could reliably set up and tear down.

An empirical study published on arXiv in 2025 found that 75% of flaky tests sit in clusters, with a mean cluster size of 13.5 across 810 flaky tests, 24 projects, and 10,000 runs. Clusters of that structure almost never point at individual test logic. They point at shared state, ordering dependencies, and environment conditions. Those problems live in the application and its infrastructure, not in the test files.

If the application has no testable seams, no way to reach a known starting point and no stable hooks for assertions, that is the first thing to address. We will say so. If the infrastructure work is in scope, we will include it. If it is not changeable within the engagement, migrating first is spending money to relocate a problem.

No owner, no lasting result

We raise this in the first conversation, not at the handover.

A migration with no named maintainer on the client side will not hold. The suite will drift from the application, failures will be silenced rather than investigated, and the state that prompted the migration will return. This is not a judgement about intent. It is a consequence of how unowned software degrades.

The actual deliverable of any migration is an engineer on your team who can diagnose a failure, extend the suite when new features ship, and decide when a failing test is wrong rather than the application. That person needs to be identified before the work starts, needs time allocated to it, and needs to have been involved in the authoring decisions, rather than handed a completed repository. If that is not possible within the engagement, we will say so and explain why.

We are glad to include structured pairing and documentation as part of scope. The goal is that the dependency on us ends.

The big-bang brief: all 3,000 tests by the quarter

A hard deadline applied to a large, untriaged corpus with no pilot batch tends to produce one of two outcomes: tests translated mechanically without being understood, or tests quietly omitted because there was no time to make them pass. Neither outcome is visible until the migration is nominally complete, and neither is useful.

The approach we would propose instead is a bounded pilot of roughly 10–15% of the suite: the highest-value flows, run through the full process including CI integration and environment configuration. That batch establishes the real complexity of the migration, surfaces infrastructure assumptions before they stall the full project, and gives the engineers who will own the result a chance to learn the shape of it.

A quarter is often enough time for a well-scoped migration. It is not enough time for an unscoped one, and no amount of pace changes that arithmetic.

Work that should stay manual

The last category is not a kind of suite but a kind of test. Exploratory testing, accessibility walkthroughs and one-off compliance checks are judgement-driven work. They depend on a person deciding whether something is right, noticing what was never specified in the first place, and adapting as they go. Encoding that judgement as an assertion removes the thing that made it valuable, and leaves you maintaining the husk.

A manual backlog is rarely a queue of things waiting to be automated. It is usually a mix: some checks that should have been automated years ago, some that are genuinely cheaper to keep doing by hand, and some that exist because nobody has questioned them since the person who wrote them left. Those three need different answers, and sorting them is most of the value of an audit.

So the second list, the one of tests we will recommend you do not automate, is usually longer than anyone expects. Paying to convert a test that needed a human to interpret it is money spent making the result worse.

What we do take on: Selenium-to-Playwright migrations on web applications, with bounded scope, testable application state, and a team ready to own the result. The free audit at hello@qualitylabs.eu will tell you which category you are in, including when the answer is that you do not need us.