Two weeks, 200 tests, and the part everyone skips

What made them move

Lingvano is an Austrian app for learning sign language. By October 2023 their end-to-end suite had grown to more than 200 Cypress tests. The team’s complaint was not that Cypress had failed catastrophically; it was four specific limitations wearing them down together.

Local runs were slow enough that engineers were isolating tests with .only markers during active development: a workflow that solves the immediate problem and quietly accumulates debt. Each marker that stays in the codebase is a test that is no longer running by default, and a suite with enough of them can drift significantly from covering the real application. Sharding, the obvious remedy for a slow suite, sat behind Cypress’s paid plan. WebKit support was listed as experimental. Lisa Weilguni, who published the team’s account on the Lingvano engineering blog in October 2023, concluded that the combination made moving worthwhile.

The AI tool that came first

Before settling on a working method, the team tried GPT-4 for the conversion. The result was described plainly in the post:

It “suggested non-existing assertions and outdated syntax.”

They abandoned that approach. This is the most useful line in the article, and it is easy to pass over.

GPT-4 did not produce garbled output. It produced plausible-looking Playwright code: code that referenced API methods formatted in the correct style, failing only because those methods do not exist. That is the failure mode that matters. A tool that generates obvious nonsense gets caught in the first review. A tool that generates confident-looking wrong code gets caught in the test run, or after merge, or not at all if the original coverage was thin.

The reason GitHub Copilot, used inline alongside Playwright’s own migration tool and third-party converters, did better is not a question of model quality. It is a question of context. Copilot, working against real files, sees the surrounding imports, the actual Playwright version in use, and the assertion patterns already present in the project. A standalone prompt to GPT-4 sees only what you put into it. The model that produces better output is frequently the one that has been given more verified context to constrain its suggestions against.

What they actually did, and why the timeline held

The final workflow combined three things: GitHub Copilot for inline suggestions as the team worked through the test files, Playwright’s own migration tool for straightforward structural transformations, and third-party converters for additional mechanical work. The team moved all 200+ tests over roughly two weeks.

Two weeks was achievable here for reasons worth naming explicitly. The suite was a known, bounded size. The team understood the application. And Cypress and Playwright are syntactically close enough that many tests translated with minimal manual work. Other published migration timelines vary enormously depending on suite age, source language, and how tightly the tests are coupled to framework internals. A migration from Selenium in Java, or from a suite built up over many years with substantial custom infrastructure, will not share these properties. Two weeks is a real result from a specific context, not a general benchmark.

The two numbers, and why you need both

The post’s title says 70%. That number comes from the local run.

Local execution fell from 1 minute 27 seconds to 25 seconds. measured

CI duration fell from 6 minutes 38 seconds to 3 minutes 43 seconds. measured

The local improvement is 71%, which the post’s title rounds to 70%. That is a defensible editorial choice for a headline. The CI improvement is 44%. The gap between those two figures is not a rounding error. It is structural.

A local run measures what the framework controls directly: process startup, browser initialisation, test execution. When Playwright handles those more efficiently than Cypress, the saving shows up in full. A CI pipeline contains all of that, plus checkout, dependency installation, build steps, artefact upload, and reporting. Playwright touches none of those. Whatever proportion of your pipeline time sits outside the test runner, that portion does not move at all. The more overhead your pipeline carries, the smaller the visible net improvement.

A manager who reads “70% faster” and builds a cost-reduction case around that figure will find a 44% outcome instead. This is a predictable consequence of measuring a framework change against a full pipeline, and it is exactly the kind of expectation gap that damages confidence in a project retrospectively. The two numbers should always be quoted together.

What the green tests did not mean

Once the suite was passing, two new problems appeared.

WebKit support (one of Playwright’s headline selling points over Cypress) turned out to be “quite slow and flaky in our GitHub CI pipeline,” in Weilguni’s words, despite working correctly on a Mac locally. This is a known pattern: Playwright’s WebKit runs on a compiled port rather than a system browser, and its behaviour on Linux CI can differ from a native macOS environment. The team gained cross-browser coverage, but they inherited maintenance overhead they had not anticipated.

The more significant constraint was Mock Service Worker. The team had been using MSW to intercept network requests in their Cypress tests. Playwright, as Weilguni notes, lacks support for MSW in its window context, which forced the team to abandon their existing MSW setup in favour of Playwright’s built-in network interception.

MSW is a good example of a wider pattern: tooling dependencies that seem peripheral until you migrate, at which point they are either unsupported or require a full replacement strategy. The useful pre-migration audit is not only the test files themselves. It is every library those tests depend on, with a clear answer for each one’s path forward in the new environment.