Update webpack from 5.97.1 to 5.98.0 causes tests stop running with component testing

[KarineBrandelli]

Description

Cypress component tests fail to run when the webpack dependency is updated from 5.97.1 to version 5.98.0 or any later version. The issue does not appear to affect end-to-end (e2e) tests in other projects, suggesting it is specific to component testing.

This error also occured to me eventually when I added some step (a new Then/When) and saved the file (not sure it helps).

Steps to reproduce

1. Acess the codesandbox and fork the repo: https://codesandbox.io/p/devbox/cypress-cucumber-3y4jwl
2. Open the terminal
3. Run yarn to install dependencies
4. Run yarn cy:run to execute the tests. The tests should pass at this stage
5. Open the package.json file
6. Change the webpack version from 5.97.1 to 5.98.0
7. In the terminal, run yarn again to update the dependencies
8. Run yarn cy:run in the terminal

Observe that the Cypress component tests now fail.

Version 5.97.1:

Version 5.98.0 (and all above)

Full error:

  1) An uncaught error was detected outside of a test:
     TypeError: The following error originated from your test code, not from Cypress.

  > Cannot read properties of null (reading 'replace')

When Cypress detects uncaught errors originating from your test code it will automatically fail the current test.

Cypress could not associate this error to any specific test.

We dynamically generated a new test to display this failure.
      at maybeRetrievePositionFromSourceMap (http://localhost:8080/__cypress/src/vendors-node_modules_badeball_cypress-cucumber-preprocessor_dist_entrypoint-browser_js-node_m-d01cc3.js:2416:39)
      at Registry.defineStep (http://localhost:8080/__cypress/src/vendors-node_modules_badeball_cypress-cucumber-preprocessor_dist_entrypoint-browser_js-node_m-d01cc3.js:2868:75)
      at defineStep (http://localhost:8080/__cypress/src/vendors-node_modules_badeball_cypress-cucumber-preprocessor_dist_entrypoint-browser_js-node_m-d01cc3.js:1594:35)
      at ./cypress/integration/common/when.step.tsx (webpack://storybook-webpack-react/./cypress/integration/common/when.step.tsx:7:0)
      at __webpack_require__ (http://localhost:8080/__cypress/src/main.js:2709:42)
      at ./cypress/integration/features/check-tree.feature (webpack://storybook-webpack-react/./cypress/integration/features/check-tree.feature:5:4)
      at __webpack_require__ (http://localhost:8080/__cypress/src/main.js:2709:42)
      at __webpack_require__.t (http://localhost:8080/__cypress/src/main.js:2757:38)

Desired behavior

The Cypress component tests should pass regardless of the webpack version (5.97.1, 5.98.0, or later). The update to webpack should not cause the tests to fail.

Cypress version

14.3.2

Preprocessor version

22.0.1

Node version

20.12.0

Operating system

Ubuntu 22.04

Checklist

• I've read the FAQ.
• I've read instructions for logging issues.
• I'm not using cypress-cucumber-preprocessor@4.3.1 (package name has changed and it is no longer the most recent version, see #689).

[badeball]

This regression is due to webpack/webpack#19184 (I bisected their changelog between v5.97.1 and v5.98.0). The new source maps resulting from said change can't (appears to) be resolved using SourceMapConsumer from source-map. Unfortunately however, I wouldn't be able to tell you why this is. I recommend that try opening up a ticket in their (webpack's) tracker.

[alexander-akait]

#badeball Hello, how do you debug this? webpack/webpack#19184 doesn't nothing with source maps

[badeball]

@alexander-akait, cloned webpack's repository, I took the example provided by OP and installed webpack from source, then bisected Webpack's changelog between v5.97.1 and v5.98.0 until I found the culprit commit.

[alexander-akait]

Very weird, I will look deeply, but order of modules should not be a problem here

[xiaoxiaojx]

This is supposed to be looking for the source mapping for stack[3], why is stack[0].fileName being passed in. fileName, column, and line should have a one-to-one correspondence. I think it might be a cypress-cucumber-preprocessor issue.
I tried that if I pass in stack[3].fileName, the code will run successfully ✅

cc @alexander-akait @KarineBrandelli

[KarineBrandelli]

This is supposed to be looking for the source mapping for stack[3], why is stack[0].fileName being passed in. fileName, column, and line should have a one-to-one correspondence. I think it might be a cypress-cucumber-preprocessor issue. I tried that if I pass in stack[3].fileName, the code will run successfully ✅ cc @alexander-akait @KarineBrandelli

This changes you made into the @badeball/cypress-cucumber-preprocessor package and tested locally?

[xiaoxiaojx]

@KarineBrandelli Yes, I tested locally. I tried that if I pass in stack[3].fileName, the code will run successfully ✅

[KarineBrandelli]

@badeball could you validate the @xiaoxiaojx solution?

[badeball]

This is supposed to be looking for the source mapping for stack[3], why is stack[0].fileName being passed in. fileName, column, and line should have a one-to-one correspondence. I think it might be a cypress-cucumber-preprocessor issue. I tried that if I pass in stack[3].fileName, the code will run successfully ✅

Ouf, you're right. This was actually intentional to begin with, as e2e tests were/are served as a single bundle, where every stack contains the same fileName attribute. However, I realize now that with the introduction of component tests, that the default webpack devserver configured by Cypress, splits code into chunks and said assumption no longer holds.

I'm going to have to spend some time testing these scenarios as well and unfortunately I don't have time right now. You (@KarineBrandelli) can temporarily work around this by disabling code splitting using the following, additional webpack config:

plugins: [
  new Webpack.optimize.LimitChunkCountPlugin({
    maxChunks: 1,
  }),
]

[KarineBrandelli]

This is supposed to be looking for the source mapping for stack[3], why is stack[0].fileName being passed in. fileName, column, and line should have a one-to-one correspondence. I think it might be a cypress-cucumber-preprocessor issue. I tried that if I pass in stack[3].fileName, the code will run successfully ✅

Ouf, you're right. This was actually intentional to begin with, as e2e tests were/are served as a single bundle, where every stack contains the same fileName attribute. However, I realize now that with the introduction of component tests, that the default webpack devserver configured by Cypress, splits code into chunks and said assumption no longer holds.

I'm going to have to spend some time testing these scenarios as well and unfortunately I don't have time right now. You (@KarineBrandelli) can temporarily work around this by disabling code splitting using the following, additional webpack config:

plugins: [
  new Webpack.optimize.LimitChunkCountPlugin({
    maxChunks: 1,
  }),
]

I'm currently on another task, but I'll test the temporarily solution later! Thanks @badeball ☺️

[badeball]

This has been fixed with v22.1.0. Please try it out and provide feedback.