Cypress.io

The most annoying test flakes are those that cannot be reproduced locally or simply don’t show up when you need them to. But we think that what happens on CI should not stay on CI.

Recording your test run using replay will provide more info than any test result, logger or any DOM snapshot trace ever could. That’s because a replay provides you with context for every line of code. For both your spec and the app you test.

Cypress brings its special chaining syntax, but can sometimes lead to unexpected issues. Retryability mechanisms can sometimes cause a race-condition, starting a chain of events that can be hard to trace back.

Replay tackles this by providing you more context and deeper insight. All packaged into DevTools that we love to work with and with time-traveling capability on top of it.

Replay allows you to track every DOM change, not just those that are captured by "before" and "after" snapshots. Let’s look at the following test:

spec.cy.ts
 1it('tests a stopwatch', () => {
 2  cy.visit('/')
 3  cy.get('p').should('have.text', '00:01:000')
 4  cy.get('p').should('have.text', '00:03:000')
 5})

There are a couple of things that could go wrong with this test:

stopwatch never shows the time 00:02:000 and still passes (false positive)
stopwatch runs too fast for .should() command and fails (false negative)

While this example is fairly exaggerated, it illustrates two main problems of rapid DOM changes.

1. they can cause problems and still be invisible for the test script
2. they can happen too fast for a test assertion to make it in time

This is where snapshots tend to fall short. Because an application is still operating between test steps, there’s a lot that goes under the radar. Replay is not a snapshot-based recorder but instead records the whole runtime. Visualized, the difference between DOM snapshots and a replay could look like this:

Because of this, a flaky test does not feel like an act of higher force, but can be inspected with a microscopic precision. Even when your app is updated every millisecond, you can jump to the perfect spot, and see your DOM rendered exactly as it would on that moment.

Replays capture all network communication and display it the same way you would expect to see it in your favorite browser’s network panel.

Network calls are the biggest contributor to asynchronous issues, as responses from server return in different times and sizes. Replay DevTools allow you to inspect every API call and resource fetch, and view headers, payloads, server responses and even timings and stack trace.

Because network panel is integrated with the rest of the tools, you’ll get even more options than in local development. Stack trace panel displays the source of an API call, which allows you to track back the very function that triggered the call.

Debugging tests should be a team effort whether that is collaboration between QA and devs or between different dev teams. When facing an API issue, commenting and sharing a replay will save hours of back and forth reporting. Instead of replicating the issue, share the issue exactly the way it happened. Locally or on CI.

When a test clicks on an element on page, we usually see the result. Modal opened, API call was made, and so on. Replay shows you how that result happened.

Interactions show "Jump to code" button that takes you to the function call made by a click, typing or other interaction. This way your test interaction is no longer disconnected from the application under test.

This bridge between your test code and your app code is usually the first place you can start your test flake investigation. It gives you a unique insight into what is happening during your test - not only with the test, but also the application under test.