Add --test-threads option to test binaries #35414
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @aturon (or someone else) soon. If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes. Please see the contribution instructions for more information. |
|
Seems reasonable to me, thanks for the PR! I'd also be fine just calling this cc @rust-lang/tools |
This change allows parallelism of test runs to be specified by a command line flag names --test-threads in addition to the existing environment variable RUST_TEST_THREADS. Fixes rust-lang#25636.
jupp0r
force-pushed
the
feature/test-threads-flag
branch
from
e36f362 to
6ca9094
Compare
|
I thought about calling the flag --threads, but decided against it because other threads might as well be spawned in tests and I wanted to avoid the impression that the flag limits the overall number of threads being created. I don't have a strong opinion, though. |
|
sgtm |
|
📌 Commit 6ca9094 has been approved by |
brson
added
the
relnotes
Manishearth
added a commit
to Manishearth/rust
that referenced
this pull request
Aug 13, 2016
…=alexcrichton Add --test-threads option to test binaries This change allows parallelism of test runs to be specified by a command line flag names --test-threads in addition to the existing environment variable RUST_TEST_THREADS. Fixes rust-lang#25636.
bors
added a commit
that referenced
this pull request
Aug 13, 2016
Add --test-threads option to test binaries This change allows parallelism of test runs to be specified by a command line flag names --test-threads in addition to the existing environment variable RUST_TEST_THREADS. Fixes #25636.
|
Is it possible to check/assert for the number of threads in a test-case ? |
epage
added a commit
to epage/rust
that referenced
this pull request
Aug 11, 2025
Like with rust-lang#139224, this is a documentation-only deprecation for now. Over time, we can - warn and then remove on use of unstable environment variables - warn on use of stable environment variables (no plan to remove due to compatibility) Longer term, we expect test runners, like `cargo test`, to provide the necessary mechanisms for environmental or persistent configuration (e.g. using cargo config which supports `.cargo/config.toml` as well as environment variables). This would include: - `RUST_TEST_THREADS` - `RUST_TEST_NOCAPTURE` - `RUST_TEST_SHUFFLE` (unstable) - `RUST_TEST_SHUFFLE_SEED` (unstable) The primary outcomes for this change are - Reducing the scope of what is expected for custom test harnesses to implement - Reduce the mechanisms that test runners, like `cargo test`, are expected to track when they are being bypassed to protect against negative interactions, e.g. `RUST_TEST_NOCAPTURE=1` when json output is being read. For testing-devex FCP, see rust-lang/testing-devex-team#10 Fixes rust-lang/testing-devex-team#10 History ------- At each step, I could not find evidence of design discussions on whether to support CLI, env, or both. The first env variable seems to come from the fact that it was being forked out of an existing env variable that had a much wider scope. At best, this seems like a way to offer a more persistent configuration for these flags but environment variables hidden away in libtest is a bit clunky and this seems like the wrong layer to handle this problem. **Originally:** `RUST_THREADS` was respected by the Rust runtime and libextra/test got this for free **2013:** rust-lang#7335 suggested splitting `RUST_TEST_TASKS` out of `RUST_THREADS`. In that issue and the implementation (rust-lang#8823). **2014:** rust-lang#13374 ask for support to disable capturing of stdout/stderr. `--nocapture` and `RUST_TEST_NOCAPTURE` were added together. **2015:** rust-lang#23525 renamed `RUST_TEST_TASKS` to `RUST_TEST_THREADS` **2016:** rust-lang#25636 asked to configure `RUST_TEST_THREADS` via `--test-threads` which was implemented in rust-lang#35414 **2021:** rust-lang#85440 asked for test randomization which was implemented in rust-lang#89082, adding `--shuffle` / RUST_TEST_SHUFFLE` and `--shuffle-seed SEED` / `RUST_TEST_SHUFFLE_SEED` Potentially relevant issues --------------------------- - rust-lang#74845
Kobzol
added a commit
to Kobzol/rust
that referenced
this pull request
Aug 12, 2025
Deprecate RUST_TEST_* env variables Like with rust-lang#139224, this is a documentation-only deprecation for now. Over time, we can - warn and then remove on use of unstable environment variables - warn on use of stable environment variables (no plan to remove due to compatibility) Longer term, we expect test runners, like `cargo test`, to provide the necessary mechanisms for environmental or persistent configuration (e.g. using cargo config which supports `.cargo/config.toml` as well as environment variables). This would include: - `RUST_TEST_THREADS` - `RUST_TEST_NOCAPTURE` - `RUST_TEST_SHUFFLE` (unstable) - `RUST_TEST_SHUFFLE_SEED` (unstable) The primary outcomes for this change are - Reducing the scope of what is expected for custom test harnesses to implement - Reduce the mechanisms that test runners, like `cargo test`, are expected to track when they are being bypassed to protect against negative interactions, e.g. `RUST_TEST_NOCAPTURE=1` when json output is being read. For testing-devex FCP, see rust-lang/testing-devex-team#10 Fixes rust-lang/testing-devex-team#10 History ------- At each step, I could not find evidence of design discussions on whether to support CLI, env, or both. The first env variable seems to come from the fact that it was being forked out of an existing env variable that had a much wider scope. At best, this seems like a way to offer a more persistent configuration for these flags but environment variables hidden away in libtest is a bit clunky and this seems like the wrong layer to handle this problem. **Originally:** `RUST_THREADS` was respected by the Rust runtime and libextra/test got this for free **2013:** rust-lang#7335 suggested splitting `RUST_TEST_TASKS` out of `RUST_THREADS`. In that issue and the implementation (rust-lang#8823). **2014:** rust-lang#13374 ask for support to disable capturing of stdout/stderr. `--nocapture` and `RUST_TEST_NOCAPTURE` were added together. **2015:** rust-lang#23525 renamed `RUST_TEST_TASKS` to `RUST_TEST_THREADS` **2016:** rust-lang#25636 asked to configure `RUST_TEST_THREADS` via `--test-threads` which was implemented in rust-lang#35414 **2021:** rust-lang#85440 asked for test randomization which was implemented in rust-lang#89082, adding `--shuffle` / RUST_TEST_SHUFFLE` and `--shuffle-seed SEED` / `RUST_TEST_SHUFFLE_SEED` Potentially relevant issues --------------------------- - rust-lang#74845
jhpratt
added a commit
to jhpratt/rust
that referenced
this pull request
Aug 12, 2025
Deprecate RUST_TEST_* env variables Like with rust-lang#139224, this is a documentation-only deprecation for now. Over time, we can - warn and then remove on use of unstable environment variables - warn on use of stable environment variables (no plan to remove due to compatibility) Longer term, we expect test runners, like `cargo test`, to provide the necessary mechanisms for environmental or persistent configuration (e.g. using cargo config which supports `.cargo/config.toml` as well as environment variables). This would include: - `RUST_TEST_THREADS` - `RUST_TEST_NOCAPTURE` - `RUST_TEST_SHUFFLE` (unstable) - `RUST_TEST_SHUFFLE_SEED` (unstable) The primary outcomes for this change are - Reducing the scope of what is expected for custom test harnesses to implement - Reduce the mechanisms that test runners, like `cargo test`, are expected to track when they are being bypassed to protect against negative interactions, e.g. `RUST_TEST_NOCAPTURE=1` when json output is being read. For testing-devex FCP, see rust-lang/testing-devex-team#10 Fixes rust-lang/testing-devex-team#10 History ------- At each step, I could not find evidence of design discussions on whether to support CLI, env, or both. The first env variable seems to come from the fact that it was being forked out of an existing env variable that had a much wider scope. At best, this seems like a way to offer a more persistent configuration for these flags but environment variables hidden away in libtest is a bit clunky and this seems like the wrong layer to handle this problem. **Originally:** `RUST_THREADS` was respected by the Rust runtime and libextra/test got this for free **2013:** rust-lang#7335 suggested splitting `RUST_TEST_TASKS` out of `RUST_THREADS`. In that issue and the implementation (rust-lang#8823). **2014:** rust-lang#13374 ask for support to disable capturing of stdout/stderr. `--nocapture` and `RUST_TEST_NOCAPTURE` were added together. **2015:** rust-lang#23525 renamed `RUST_TEST_TASKS` to `RUST_TEST_THREADS` **2016:** rust-lang#25636 asked to configure `RUST_TEST_THREADS` via `--test-threads` which was implemented in rust-lang#35414 **2021:** rust-lang#85440 asked for test randomization which was implemented in rust-lang#89082, adding `--shuffle` / RUST_TEST_SHUFFLE` and `--shuffle-seed SEED` / `RUST_TEST_SHUFFLE_SEED` Potentially relevant issues --------------------------- - rust-lang#74845
jhpratt
added a commit
to jhpratt/rust
that referenced
this pull request
Aug 12, 2025
Deprecate RUST_TEST_* env variables Like with rust-lang#139224, this is a documentation-only deprecation for now. Over time, we can - warn and then remove on use of unstable environment variables - warn on use of stable environment variables (no plan to remove due to compatibility) Longer term, we expect test runners, like `cargo test`, to provide the necessary mechanisms for environmental or persistent configuration (e.g. using cargo config which supports `.cargo/config.toml` as well as environment variables). This would include: - `RUST_TEST_THREADS` - `RUST_TEST_NOCAPTURE` - `RUST_TEST_SHUFFLE` (unstable) - `RUST_TEST_SHUFFLE_SEED` (unstable) The primary outcomes for this change are - Reducing the scope of what is expected for custom test harnesses to implement - Reduce the mechanisms that test runners, like `cargo test`, are expected to track when they are being bypassed to protect against negative interactions, e.g. `RUST_TEST_NOCAPTURE=1` when json output is being read. For testing-devex FCP, see rust-lang/testing-devex-team#10 Fixes rust-lang/testing-devex-team#10 History ------- At each step, I could not find evidence of design discussions on whether to support CLI, env, or both. The first env variable seems to come from the fact that it was being forked out of an existing env variable that had a much wider scope. At best, this seems like a way to offer a more persistent configuration for these flags but environment variables hidden away in libtest is a bit clunky and this seems like the wrong layer to handle this problem. **Originally:** `RUST_THREADS` was respected by the Rust runtime and libextra/test got this for free **2013:** rust-lang#7335 suggested splitting `RUST_TEST_TASKS` out of `RUST_THREADS`. In that issue and the implementation (rust-lang#8823). **2014:** rust-lang#13374 ask for support to disable capturing of stdout/stderr. `--nocapture` and `RUST_TEST_NOCAPTURE` were added together. **2015:** rust-lang#23525 renamed `RUST_TEST_TASKS` to `RUST_TEST_THREADS` **2016:** rust-lang#25636 asked to configure `RUST_TEST_THREADS` via `--test-threads` which was implemented in rust-lang#35414 **2021:** rust-lang#85440 asked for test randomization which was implemented in rust-lang#89082, adding `--shuffle` / RUST_TEST_SHUFFLE` and `--shuffle-seed SEED` / `RUST_TEST_SHUFFLE_SEED` Potentially relevant issues --------------------------- - rust-lang#74845
jhpratt
added a commit
to jhpratt/rust
that referenced
this pull request
Aug 12, 2025
Deprecate RUST_TEST_* env variables Like with rust-lang#139224, this is a documentation-only deprecation for now. Over time, we can - warn and then remove on use of unstable environment variables - warn on use of stable environment variables (no plan to remove due to compatibility) Longer term, we expect test runners, like `cargo test`, to provide the necessary mechanisms for environmental or persistent configuration (e.g. using cargo config which supports `.cargo/config.toml` as well as environment variables). This would include: - `RUST_TEST_THREADS` - `RUST_TEST_NOCAPTURE` - `RUST_TEST_SHUFFLE` (unstable) - `RUST_TEST_SHUFFLE_SEED` (unstable) The primary outcomes for this change are - Reducing the scope of what is expected for custom test harnesses to implement - Reduce the mechanisms that test runners, like `cargo test`, are expected to track when they are being bypassed to protect against negative interactions, e.g. `RUST_TEST_NOCAPTURE=1` when json output is being read. For testing-devex FCP, see rust-lang/testing-devex-team#10 Fixes rust-lang/testing-devex-team#10 History ------- At each step, I could not find evidence of design discussions on whether to support CLI, env, or both. The first env variable seems to come from the fact that it was being forked out of an existing env variable that had a much wider scope. At best, this seems like a way to offer a more persistent configuration for these flags but environment variables hidden away in libtest is a bit clunky and this seems like the wrong layer to handle this problem. **Originally:** `RUST_THREADS` was respected by the Rust runtime and libextra/test got this for free **2013:** rust-lang#7335 suggested splitting `RUST_TEST_TASKS` out of `RUST_THREADS`. In that issue and the implementation (rust-lang#8823). **2014:** rust-lang#13374 ask for support to disable capturing of stdout/stderr. `--nocapture` and `RUST_TEST_NOCAPTURE` were added together. **2015:** rust-lang#23525 renamed `RUST_TEST_TASKS` to `RUST_TEST_THREADS` **2016:** rust-lang#25636 asked to configure `RUST_TEST_THREADS` via `--test-threads` which was implemented in rust-lang#35414 **2021:** rust-lang#85440 asked for test randomization which was implemented in rust-lang#89082, adding `--shuffle` / RUST_TEST_SHUFFLE` and `--shuffle-seed SEED` / `RUST_TEST_SHUFFLE_SEED` Potentially relevant issues --------------------------- - rust-lang#74845
Kobzol
added a commit
to Kobzol/rust
that referenced
this pull request
Aug 13, 2025
Deprecate RUST_TEST_* env variables Like with rust-lang#139224, this is a documentation-only deprecation for now. Over time, we can - warn and then remove on use of unstable environment variables - warn on use of stable environment variables (no plan to remove due to compatibility) Longer term, we expect test runners, like `cargo test`, to provide the necessary mechanisms for environmental or persistent configuration (e.g. using cargo config which supports `.cargo/config.toml` as well as environment variables). This would include: - `RUST_TEST_THREADS` - `RUST_TEST_NOCAPTURE` - `RUST_TEST_SHUFFLE` (unstable) - `RUST_TEST_SHUFFLE_SEED` (unstable) The primary outcomes for this change are - Reducing the scope of what is expected for custom test harnesses to implement - Reduce the mechanisms that test runners, like `cargo test`, are expected to track when they are being bypassed to protect against negative interactions, e.g. `RUST_TEST_NOCAPTURE=1` when json output is being read. For testing-devex FCP, see rust-lang/testing-devex-team#10 Fixes rust-lang/testing-devex-team#10 History ------- At each step, I could not find evidence of design discussions on whether to support CLI, env, or both. The first env variable seems to come from the fact that it was being forked out of an existing env variable that had a much wider scope. At best, this seems like a way to offer a more persistent configuration for these flags but environment variables hidden away in libtest is a bit clunky and this seems like the wrong layer to handle this problem. **Originally:** `RUST_THREADS` was respected by the Rust runtime and libextra/test got this for free **2013:** rust-lang#7335 suggested splitting `RUST_TEST_TASKS` out of `RUST_THREADS`. In that issue and the implementation (rust-lang#8823). **2014:** rust-lang#13374 ask for support to disable capturing of stdout/stderr. `--nocapture` and `RUST_TEST_NOCAPTURE` were added together. **2015:** rust-lang#23525 renamed `RUST_TEST_TASKS` to `RUST_TEST_THREADS` **2016:** rust-lang#25636 asked to configure `RUST_TEST_THREADS` via `--test-threads` which was implemented in rust-lang#35414 **2021:** rust-lang#85440 asked for test randomization which was implemented in rust-lang#89082, adding `--shuffle` / RUST_TEST_SHUFFLE` and `--shuffle-seed SEED` / `RUST_TEST_SHUFFLE_SEED` Potentially relevant issues --------------------------- - rust-lang#74845
rust-timer
added a commit
that referenced
this pull request
Aug 13, 2025
Rollup merge of #145269 - epage:test-env, r=jhpratt Deprecate RUST_TEST_* env variables Like with #139224, this is a documentation-only deprecation for now. Over time, we can - warn and then remove on use of unstable environment variables - warn on use of stable environment variables (no plan to remove due to compatibility) Longer term, we expect test runners, like `cargo test`, to provide the necessary mechanisms for environmental or persistent configuration (e.g. using cargo config which supports `.cargo/config.toml` as well as environment variables). This would include: - `RUST_TEST_THREADS` - `RUST_TEST_NOCAPTURE` - `RUST_TEST_SHUFFLE` (unstable) - `RUST_TEST_SHUFFLE_SEED` (unstable) The primary outcomes for this change are - Reducing the scope of what is expected for custom test harnesses to implement - Reduce the mechanisms that test runners, like `cargo test`, are expected to track when they are being bypassed to protect against negative interactions, e.g. `RUST_TEST_NOCAPTURE=1` when json output is being read. For testing-devex FCP, see rust-lang/testing-devex-team#10 Fixes rust-lang/testing-devex-team#10 History ------- At each step, I could not find evidence of design discussions on whether to support CLI, env, or both. The first env variable seems to come from the fact that it was being forked out of an existing env variable that had a much wider scope. At best, this seems like a way to offer a more persistent configuration for these flags but environment variables hidden away in libtest is a bit clunky and this seems like the wrong layer to handle this problem. **Originally:** `RUST_THREADS` was respected by the Rust runtime and libextra/test got this for free **2013:** #7335 suggested splitting `RUST_TEST_TASKS` out of `RUST_THREADS`. In that issue and the implementation (#8823). **2014:** #13374 ask for support to disable capturing of stdout/stderr. `--nocapture` and `RUST_TEST_NOCAPTURE` were added together. **2015:** #23525 renamed `RUST_TEST_TASKS` to `RUST_TEST_THREADS` **2016:** #25636 asked to configure `RUST_TEST_THREADS` via `--test-threads` which was implemented in #35414 **2021:** #85440 asked for test randomization which was implemented in #89082, adding `--shuffle` / RUST_TEST_SHUFFLE` and `--shuffle-seed SEED` / `RUST_TEST_SHUFFLE_SEED` Potentially relevant issues --------------------------- - #74845
github-actions bot
pushed a commit
to model-checking/verify-rust-std
that referenced
this pull request
Aug 18, 2025
Deprecate RUST_TEST_* env variables Like with rust-lang#139224, this is a documentation-only deprecation for now. Over time, we can - warn and then remove on use of unstable environment variables - warn on use of stable environment variables (no plan to remove due to compatibility) Longer term, we expect test runners, like `cargo test`, to provide the necessary mechanisms for environmental or persistent configuration (e.g. using cargo config which supports `.cargo/config.toml` as well as environment variables). This would include: - `RUST_TEST_THREADS` - `RUST_TEST_NOCAPTURE` - `RUST_TEST_SHUFFLE` (unstable) - `RUST_TEST_SHUFFLE_SEED` (unstable) The primary outcomes for this change are - Reducing the scope of what is expected for custom test harnesses to implement - Reduce the mechanisms that test runners, like `cargo test`, are expected to track when they are being bypassed to protect against negative interactions, e.g. `RUST_TEST_NOCAPTURE=1` when json output is being read. For testing-devex FCP, see rust-lang/testing-devex-team#10 Fixes rust-lang/testing-devex-team#10 History ------- At each step, I could not find evidence of design discussions on whether to support CLI, env, or both. The first env variable seems to come from the fact that it was being forked out of an existing env variable that had a much wider scope. At best, this seems like a way to offer a more persistent configuration for these flags but environment variables hidden away in libtest is a bit clunky and this seems like the wrong layer to handle this problem. **Originally:** `RUST_THREADS` was respected by the Rust runtime and libextra/test got this for free **2013:** rust-lang#7335 suggested splitting `RUST_TEST_TASKS` out of `RUST_THREADS`. In that issue and the implementation (rust-lang#8823). **2014:** rust-lang#13374 ask for support to disable capturing of stdout/stderr. `--nocapture` and `RUST_TEST_NOCAPTURE` were added together. **2015:** rust-lang#23525 renamed `RUST_TEST_TASKS` to `RUST_TEST_THREADS` **2016:** rust-lang#25636 asked to configure `RUST_TEST_THREADS` via `--test-threads` which was implemented in rust-lang#35414 **2021:** rust-lang#85440 asked for test randomization which was implemented in rust-lang#89082, adding `--shuffle` / RUST_TEST_SHUFFLE` and `--shuffle-seed SEED` / `RUST_TEST_SHUFFLE_SEED` Potentially relevant issues --------------------------- - rust-lang#74845
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change allows parallelism of test runs to be specified by a
command line flag named --test-threads in addition to the existing
environment variable RUST_TEST_THREADS. Fixes #25636.