feat(@jest/transform)!: require process() and processAsync() methods to always return structured data (#12638)

Author: mrazauskas

patches/react-native.patch renamed to .yarn/patches/react-native-npm-0.68.0-9eb3ecb60a.patch

@@ -1,8 +1,17 @@
 diff --git a/jest/preprocessor.js b/jest/preprocessor.js
-index 5920c0a6f23c056f27366fabf32dd13c6f86465b..2658e52f9127ac58849e7f830f6342d8b683672c 100644
+index f5e34763d840193e243a974e448b619f8f635095..cc6e05ab4c9c25a3d257379a1bf19c21fb9ef5ca 100644
 --- a/jest/preprocessor.js
 +++ b/jest/preprocessor.js
-@@ -64,8 +64,6 @@ module.exports = {
+@@ -38,7 +38,7 @@ module.exports = {
+         sourceType: 'script',
+         ...nodeOptions,
+         ast: false,
+-      }).code;
++      });
+     }
+ 
+     const {ast} = transformer.transform({
+@@ -66,8 +66,6 @@ module.exports = {
          [require('@babel/plugin-transform-flow-strip-types')],
          [
            require('@babel/plugin-proposal-class-properties'),
@@ -11,3 +20,12 @@ index 5920c0a6f23c056f27366fabf32dd13c6f86465b..2658e52f9127ac58849e7f830f6342d8
          ],
          [require('@babel/plugin-transform-computed-properties')],
          [require('@babel/plugin-transform-destructuring')],
+@@ -112,7 +110,7 @@ module.exports = {
+         sourceMaps: true,
+       },
+       src,
+-    ).code;
++    );
+   },
+ 
+   getCacheKey: (createCacheKeyFunction([

CHANGELOG.md

@@ -43,6 +43,7 @@
 - `[jest-runtime]` [**BREAKING**] `Runtime.createHasteMap` now returns a promise ([#12008](https://github.com/facebook/jest/pull/12008))
 - `[jest-runtime]` Calling `jest.resetModules` function will clear FS and transform cache ([#12531](https://github.com/facebook/jest/pull/12531))
 - `[@jest/schemas]` New module for JSON schemas for Jest's config ([#12384](https://github.com/facebook/jest/pull/12384))
+- `[jest-transform]` [**BREAKING**] Make it required for `process()` and `processAsync()` methods to always return structured data ([#12638](https://github.com/facebook/jest/pull/12638))
 - `[jest-test-result]` Add duration property to JSON test output ([#12518](https://github.com/facebook/jest/pull/12518))
 - `[jest-watcher]` [**BREAKING**] Make `PatternPrompt` class to take `entityName` as third constructor parameter instead of `this._entityName` ([#12591](https://github.com/facebook/jest/pull/12591))
 - `[jest-worker]` [**BREAKING**] Allow only absolute `workerPath` ([#12343](https://github.com/facebook/jest/pull/12343))

docs/CodeTransformation.md

@@ -3,29 +3,36 @@ id: code-transformation
 title: Code Transformation
 ---
 
-Jest runs the code in your project as JavaScript, but if you use some syntax not supported by Node.js out of the box (such as JSX, types from TypeScript, Vue templates etc.) then you'll need to transform that code into plain JavaScript, similar to what you would do when building for browsers.
+Jest runs the code in your project as JavaScript, but if you use some syntax not supported by Node out of the box (such as JSX, TypeScript, Vue templates) then you'll need to transform that code into plain JavaScript, similar to what you would do when building for browsers.
 
-Jest supports this via the [`transform` configuration option](Configuration.md#transform-objectstring-pathtotransformer--pathtotransformer-object).
+Jest supports this via the [`transform`](Configuration.md#transform-objectstring-pathtotransformer--pathtotransformer-object) configuration option.
 
-A transformer is a module that provides a synchronous function for transforming source files. For example, if you wanted to be able to use a new language feature in your modules or tests that aren't yet supported by Node, you might plug in one of many compilers that compile a future version of JavaScript to a current one.
+A transformer is a module that provides a method for transforming source files. For example, if you wanted to be able to use a new language feature in your modules or tests that aren't yet supported by Node, you might plug in a code preprocessor that would transpile a future version of JavaScript to a current one.
 
 Jest will cache the result of a transformation and attempt to invalidate that result based on a number of factors, such as the source of the file being transformed and changing configuration.
 
 ## Defaults
 
-Jest ships with one transformer out of the box - `babel-jest`. It will automatically load your project's Babel configuration and transform any file matching the following RegEx: `/\.[jt]sx?$/` meaning any `.js`, `.jsx`, `.ts` and `.tsx` file. In addition, `babel-jest` will inject the Babel plugin necessary for mock hoisting talked about in [ES Module mocking](ManualMocks.md#using-with-es-module-imports).
+Jest ships with one transformer out of the box – [`babel-jest`](https://github.com/facebook/jest/tree/main/packages/babel-jest#setup). It will load your project's Babel configuration and transform any file matching the `/\.[jt]sx?$/` RegExp (in other words, any `.js`, `.jsx`, `.ts` or `.tsx` file). In addition, `babel-jest` will inject the Babel plugin necessary for mock hoisting talked about in [ES Module mocking](ManualMocks.md#using-with-es-module-imports).
 
-If you override the `transform` configuration option `babel-jest` will no longer be active, and you'll need to add it manually if you wish to use Babel.
+:::tip
+
+Remember to include the default `babel-jest` transformer explicitly, if you wish to use it alongside with additional code preprocessors:
+
+```json
+"transform": {
+  "\\.[jt]sx?$": "babel-jest",
+  "\\.css$": "some-css-transformer",
+}
+```
+
+:::
 
 ## Writing custom transformers
 
 You can write your own transformer. The API of a transformer is as follows:
 
 ```ts
-// This version of the interface you are seeing on the website has been trimmed down for brevity
-// For the full definition, see `packages/jest-transform/src/types.ts` in https://github.com/facebook/jest
-// (taking care in choosing the right tag/commit for your version of Jest)
-
 interface TransformOptions<OptionType = unknown> {
   supportsDynamicImport: boolean;
   supportsExportNamespaceFrom: boolean;
@@ -41,6 +48,11 @@ interface TransformOptions<OptionType = unknown> {
   transformerConfig: OptionType;
 }
 
+type TransformedSource = {
+  code: string;
+  map?: RawSourceMap | string | null;
+};
+
 interface SyncTransformer<OptionType = unknown> {
   canInstrument?: boolean;
 
@@ -111,6 +123,12 @@ type TransformerFactory<X extends Transformer> = {
 };
 ```
 
+:::note
+
+The definitions above were trimmed down for brevity. Full code can be found in [Jest repo on GitHub](https://github.com/facebook/jest/blob/main/packages/jest-transform/src/types.ts) (remember to choose the right tag/commit for your version of Jest).
+
+:::
+
 There are a couple of ways you can import code into Jest - using Common JS (`require`) or ECMAScript Modules (`import` - which exists in static and dynamic versions). Jest passes files through code transformation on demand (for instance when a `require` or `import` is evaluated). This process, also known as "transpilation", might happen _synchronously_ (in the case of `require`), or _asynchronously_ (in the case of `import` or `import()`, the latter of which also works from Common JS modules). For this reason, the interface exposes both pairs of methods for asynchronous and synchronous processes: `process{Async}` and `getCacheKey{Async}`. The latter is called to figure out if we need to call `process{Async}` at all. Since async transformation can happen synchronously without issue, it's possible for the async case to "fall back" to the sync variant, but not vice versa.
 
 So if your code base is ESM only implementing the async variants is sufficient. Otherwise, if any code is loaded through `require` (including `createRequire` from within ESM), then you need to implement the synchronous variant. Be aware that `node_modules` is not transpiled with default config.
@@ -125,7 +143,9 @@ Note that [ECMAScript module](ECMAScriptModules.md) support is indicated by the
 
 :::tip
 
-Make sure `TransformedSource` contains a source map, so it is possible to report line information accurately in code coverage and test errors. Inline source maps also work but are slower.
+Make sure `process{Async}` method returns source map alongside with transformed code, so it is possible to report line information accurately in code coverage and test errors. Inline source maps also work but are slower.
+
+During the development of a transformer it can be useful to run Jest with `--no-cache` to frequently [delete cache](Troubleshooting.md#caching-issues).
 
 :::
 
@@ -143,8 +163,10 @@ Importing images is a way to include them in your browser bundle, but they are n
 const path = require('path');
 
 module.exports = {
-  process(src, filename, config, options) {
-    return `module.exports = ${JSON.stringify(path.basename(filename))};`;
+  process(sourceText, sourcePath, options) {
+    return {
+      code: `module.exports = ${JSON.stringify(path.basename(sourcePath))};`,
+    };
   },
 };
 ```

docs/Configuration.md

@@ -1564,27 +1564,26 @@ Default timeout of a test in milliseconds.
 
 Default: `{"\\.[jt]sx?$": "babel-jest"}`
 
-A map from regular expressions to paths to transformers. A transformer is a module that provides a synchronous function for transforming source files. For example, if you wanted to be able to use a new language feature in your modules or tests that aren't yet supported by node, you might plug in one of many compilers that compile a future version of JavaScript to a current one. Example: see the [examples/typescript](https://github.com/facebook/jest/blob/main/examples/typescript/package.json#L16) example or the [webpack tutorial](Webpack.md).
+A map from regular expressions to paths to transformers. Optionally, a tuple with configuration options can be passed as second argument: `{filePattern: ['path-to-transformer', {options}]}`. For example, here is how you can configure `babel-jest` for non-default behavior: `{'\\.js$': ['babel-jest', {rootMode: 'upward'}]}`.
 
-Examples of such compilers include:
+Jest runs the code of your project as JavaScript, hence a transformer is needed if you use some syntax not supported by Node out of the box (such as JSX, TypeScript, Vue templates). By default, Jest will use [`babel-jest`](https://github.com/facebook/jest/tree/main/packages/babel-jest#setup) transformer, which will load your project's Babel configuration and transform any file matching the `/\.[jt]sx?$/` RegExp (in other words, any `.js`, `.jsx`, `.ts` or `.tsx` file). In addition, `babel-jest` will inject the Babel plugin necessary for mock hoisting talked about in [ES Module mocking](ManualMocks.md#using-with-es-module-imports).
 
-- [Babel](https://babeljs.io/)
-- [TypeScript](http://www.typescriptlang.org/)
-- To build your own please visit the [Custom Transformer](CodeTransformation.md#writing-custom-transformers) section
-
-You can pass configuration to a transformer like `{filePattern: ['path-to-transformer', {options}]}` For example, to configure babel-jest for non-default behavior, `{"\\.js$": ['babel-jest', {rootMode: "upward"}]}`
+See the [Code Transformation](CodeTransformation.md) section for more details and instructions on building your own transformer.
 
 :::tip
 
-A transformer is only run once per file unless the file has changed. During the development of a transformer it can be useful to run Jest with `--no-cache` to frequently [delete Jest's cache](Troubleshooting.md#caching-issues).
-
-When adding additional code transformers, this will overwrite the default config and `babel-jest` is no longer automatically loaded. If you want to use it to compile JavaScript or TypeScript, it has to be explicitly defined by adding `{"\\.[jt]sx?$": "babel-jest"}` to the transform property. See [babel-jest plugin](https://github.com/facebook/jest/tree/main/packages/babel-jest#setup).
+Keep in mind that a transformer only runs once per file unless the file has changed.
 
-:::
+Remember to include the default `babel-jest` transformer explicitly, if you wish to use it alongside with additional code preprocessors:
 
-A transformer must be an object with at least a `process` function, and it's also recommended to include a `getCacheKey` function. If your transformer is written in ESM you should have a default export with that object.
+```json
+"transform": {
+  "\\.[jt]sx?$": "babel-jest",
+  "\\.css$": "some-css-transformer",
+}
+```
 
-If the tests are written using [native ESM](ECMAScriptModules.md) the transformer can export `processAsync` and `getCacheKeyAsync` instead or in addition to the synchronous variants.
+:::
 
 ### `transformIgnorePatterns` \[array<string>]
 

docs/Webpack.md

@@ -83,16 +83,16 @@ Then all your className lookups on the styles object will be returned as-is (e.g
 }
 ```
 
-> Notice that Proxy is enabled in Node 6 by default. If you are not on Node 6 yet, make sure you invoke Jest using `node --harmony_proxies node_modules/.bin/jest`.
-
 If `moduleNameMapper` cannot fulfill your requirements, you can use Jest's [`transform`](Configuration.md#transform-objectstring-pathtotransformer--pathtotransformer-object) config option to specify how assets are transformed. For example, a transformer that returns the basename of a file (such that `require('logo.jpg');` returns `'logo'`) can be written as:
 
 ```js title="fileTransformer.js"
 const path = require('path');
 
 module.exports = {
-  process(src, filename, config, options) {
-    return `module.exports = ${JSON.stringify(path.basename(filename))};`;
+  process(sourceText, sourcePath, options) {
+    return {
+      code: `module.exports = ${JSON.stringify(path.basename(sourcePath))};`,
+    };
   },
 };
 ```
@@ -112,16 +112,19 @@ module.exports = {
 
 We've told Jest to ignore files matching a stylesheet or image extension, and instead, require our mock files. You can adjust the regular expression to match the file types your webpack config handles.
 
-_Note: if you are using babel-jest with additional code preprocessors, you have to explicitly define babel-jest as a transformer for your JavaScript code to map `.js` files to the babel-jest module._
+:::tip
+
+Remember to include the default `babel-jest` transformer explicitly, if you wish to use it alongside with additional code preprocessors:
 
 ```json
 "transform": {
-  "\\.js$": "babel-jest",
-  "\\.css$": "custom-transformer",
-  ...
+  "\\.[jt]sx?$": "babel-jest",
+  "\\.css$": "some-css-transformer",
 }
 ```
 
+:::
+
 ### Configuring Jest to find our files
 
 Now that Jest knows how to process our files, we need to tell it how to _find_ them. For webpack's `modulesDirectories`, and `extensions` options there are direct analogs in Jest's `moduleDirectories` and `moduleFileExtensions` options.
@@ -186,8 +189,7 @@ That's it! webpack is a complex and flexible tool, so you may have to make some
 
 webpack 2 offers native support for ES modules. However, Jest runs in Node, and thus requires ES modules to be transpiled to CommonJS modules. As such, if you are using webpack 2, you most likely will want to configure Babel to transpile ES modules to CommonJS modules only in the `test` environment.
 
-```json
-// .babelrc
+```json title=".babelrc"
 {
   "presets": [["env", {"modules": false}]],
 
@@ -203,8 +205,7 @@ webpack 2 offers native support for ES modules. However, Jest runs in Node, and
 
 If you use dynamic imports (`import('some-file.js').then(module => ...)`), you need to enable the `dynamic-import-node` plugin.
 
-```json
-// .babelrc
+```json title=".babelrc"
 {
   "presets": [["env", {"modules": false}]],
 

e2e/__tests__/dependencyClash.test.ts

@@ -64,10 +64,12 @@ test('does not require project modules from inside node_modules', () => {
           if (!threw) {
             throw new Error('It used the wrong invariant module!');
           }
-          return script.replace(
-            'INVALID CODE FRAGMENT THAT WILL BE REMOVED BY THE TRANSFORMER',
-            ''
-          );
+          return {
+            code: script.replace(
+              'INVALID CODE FRAGMENT THAT WILL BE REMOVED BY THE TRANSFORMER',
+              '',
+            ),
+          };
         },
       };
     `,

e2e/__tests__/multiProjectRunner.test.ts

@@ -450,7 +450,7 @@ test('Does transform files with the corresponding project transformer', () => {
       };`,
     'project1/transformer.js': `
       module.exports = {
-        process: () => 'module.exports = "PROJECT1";',
+        process: () => ({code: 'module.exports = "PROJECT1";'}),
         getCacheKey: () => 'PROJECT1_CACHE_KEY',
       }
     `,
@@ -465,7 +465,7 @@ test('Does transform files with the corresponding project transformer', () => {
       };`,
     'project2/transformer.js': `
       module.exports = {
-        process: () => 'module.exports = "PROJECT2";',
+        process: () => ({code: 'module.exports = "PROJECT2";'}),
         getCacheKey: () => 'PROJECT2_CACHE_KEY',
       }
     `,

e2e/coverage-provider-v8/esm-with-custom-transformer/typescriptPreprocessor.js

@@ -21,6 +21,6 @@ export default {
 
       return {code: outputText, map: sourceMapText};
     }
-    return sourceText;
+    return {code: sourceText};
   },
 };

e2e/coverage-provider-v8/no-sourcemap/cssTransform.js

@@ -7,5 +7,5 @@
 
 module.exports = {
   getCacheKey: () => 'cssTransform',
-  process: () => 'module.exports = {};',
+  process: () => ({code: 'module.exports = {};'}),
 };

e2e/coverage-remapping/typescriptPreprocessor.js

@@ -22,6 +22,6 @@ module.exports = {
         map: JSON.parse(result.sourceMapText),
       };
     }
-    return src;
+    return {code: src};
   },
 };