compiletest: Implement an experimental `--new-output-capture` mode #146119

Zalathar · 2025-09-02T10:29:28Z

Thanks to the efforts on #140192, compiletest no longer has an unstable dependency on libtest, but it still has an unstable dependency on #![feature(internal_output_capture)]. That makes building compiletest more complicated than for most other bootstrap tools.

This PR therefore adds opt-in support for an experimental compiletest mode that avoids the use of internal_output_capture APIs, and instead uses more mundane means to capture the output of individual test runners.

Each TestCx now contains &dyn ConsoleOut references for stdout and stderr. All print statements in compiletests::runtest have been replaced with write! or writeln! calls that explicitly write to one of those trait objects. The underlying implementation then forwards to print! or eprint! (for --no-capture or old-output-capture mode), or writes to a separate buffer (in new-output-capture mode).

Currently, new-output-capture is disabled by default. It can be explicitly enabled in one of two ways:

When running x test, pass --new-output-capture=on as a compiletest argument (after --).
- E.g. x test ui -- --new-output-capture=on.
- The short form is -Non or -Ny.
Set environment variable COMPILETEST_NEW_OUTPUT_CAPTURE=on.

After some amount of opt-in testing, new-output-capture will become the default (with a temporary opt-out). Eventually, old-output-capture and #![feature(internal_output_capture)] will be completely removed from compiletest.

r? jieyouxu

rustbot · 2025-09-02T10:29:35Z

Some changes occurred in src/tools/compiletest

cc @jieyouxu

This PR modifies bootstrap.example.toml.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

Kobzol · 2025-09-02T10:57:08Z

I think that if this is an experimental feature designed for testing with a handful of people, I wouldn't add it to bootstrap.toml (nor add a change tracker entry), and just use some magic environment variable for it. bootstrap.toml should ideally be used for configuration that will last long-term, as it's rather annoying to remove stuff from it.

Zalathar · 2025-09-02T11:11:27Z

Fair enough; I'll switch over to an environment variable instead.

jieyouxu

Thanks for working on this! Other changes look good to me, just a tiny nit regarding "please don't support so many ways to spell a true/false" of the flag lol

View changes since this review

src/tools/compiletest/src/lib.rs

jieyouxu · 2025-09-03T13:35:49Z

@rustbot author

rustbot · 2025-09-03T13:35:53Z

Reminder, once the PR becomes ready for a review, use @rustbot ready.

jieyouxu · 2025-09-04T00:46:29Z

@bors r+ rollup

bors · 2025-09-04T00:46:32Z

📌 Commit 9c5e1cb has been approved by jieyouxu

It is now in the queue for this repository.

Zalathar · 2025-09-04T02:53:03Z

Added a short comment to parse_bool_option, since the topic of which values to accept came up in review.

@bors r=jieyouxu

bors · 2025-09-04T02:53:06Z

📌 Commit c2ae2e0 has been approved by jieyouxu

It is now in the queue for this repository.

compiletest: Implement an experimental `--new-output-capture` mode Thanks to the efforts on rust-lang#140192, compiletest no longer has an unstable dependency on libtest, but it still has an unstable dependency on `#![feature(internal_output_capture)]`. That makes building compiletest more complicated than for most other bootstrap tools. This PR therefore adds opt-in support for an experimental compiletest mode that avoids the use of `internal_output_capture` APIs, and instead uses more mundane means to capture the output of individual test runners. Each `TestCx` now contains `&dyn ConsoleOut` references for stdout and stderr. All print statements in `compiletests::runtest` have been replaced with `write!` or `writeln!` calls that explicitly write to one of those trait objects. The underlying implementation then forwards to `print!` or `eprint!` (for `--no-capture` or old-output-capture mode), or writes to a separate buffer (in new-output-capture mode). --- Currently, new-output-capture is disabled by default. It can be explicitly enabled in one of two ways: - When running `x test`, pass `--new-output-capture=on` as a *compiletest* argument (after `--`). - E.g. `x test ui -- --new-output-capture=on`. - The short form is `-Non` or `-Ny`. - Set environment variable `COMPILETEST_NEW_OUTPUT_CAPTURE=on`. After some amount of opt-in testing, new-output-capture will become the default (with a temporary opt-out). Eventually, old-output-capture and `#![feature(internal_output_capture)]` will be completely removed from compiletest. r? jieyouxu

Rollup of 5 pull requests Successful merges: - #145682 (Promote aarch64-pc-windows-msvc to Tier 1) - #145690 (Implement Integer funnel shifts) - #146119 (compiletest: Implement an experimental `--new-output-capture` mode) - #146168 (Update bootstrap's dependencies to remove winapi and old windows-sys) - #146182 (Don't require next-solver `ProbeRef` to be `Copy`) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of #146119 - Zalathar:capture, r=jieyouxu compiletest: Implement an experimental `--new-output-capture` mode Thanks to the efforts on #140192, compiletest no longer has an unstable dependency on libtest, but it still has an unstable dependency on `#![feature(internal_output_capture)]`. That makes building compiletest more complicated than for most other bootstrap tools. This PR therefore adds opt-in support for an experimental compiletest mode that avoids the use of `internal_output_capture` APIs, and instead uses more mundane means to capture the output of individual test runners. Each `TestCx` now contains `&dyn ConsoleOut` references for stdout and stderr. All print statements in `compiletests::runtest` have been replaced with `write!` or `writeln!` calls that explicitly write to one of those trait objects. The underlying implementation then forwards to `print!` or `eprint!` (for `--no-capture` or old-output-capture mode), or writes to a separate buffer (in new-output-capture mode). --- Currently, new-output-capture is disabled by default. It can be explicitly enabled in one of two ways: - When running `x test`, pass `--new-output-capture=on` as a *compiletest* argument (after `--`). - E.g. `x test ui -- --new-output-capture=on`. - The short form is `-Non` or `-Ny`. - Set environment variable `COMPILETEST_NEW_OUTPUT_CAPTURE=on`. After some amount of opt-in testing, new-output-capture will become the default (with a temporary opt-out). Eventually, old-output-capture and `#![feature(internal_output_capture)]` will be completely removed from compiletest. r? jieyouxu

Rollup of 5 pull requests Successful merges: - rust-lang/rust#145682 (Promote aarch64-pc-windows-msvc to Tier 1) - rust-lang/rust#145690 (Implement Integer funnel shifts) - rust-lang/rust#146119 (compiletest: Implement an experimental `--new-output-capture` mode) - rust-lang/rust#146168 (Update bootstrap's dependencies to remove winapi and old windows-sys) - rust-lang/rust#146182 (Don't require next-solver `ProbeRef` to be `Copy`) r? `@ghost` `@rustbot` modify labels: rollup

Zalathar · 2025-09-12T03:28:47Z

I was thinking of making this the default after a week had passed, but maybe I should wait another week to dodge the release window and nightly-to-beta uplift.

(Arguably I'm being overly cautious, but also we're in no great hurry.)

compiletest: Enable new-output-capture by default The new output-capture implementation was added in rust-lang#146119, but was disabled by default and required opt-in. Since then, I haven't encountered any problems in my own testing/usage, and I haven't heard any problem reports from other contributors who might have opted in. It's unlikely that more opt-in testing will help, so the next step is to enable new-output-capture by default and see if anyone complains. (Hopefully nobody!) If needed, the new default can be overridden (for now) by setting environment variable `COMPILETEST_NEW_OUTPUT_CAPTURE=off`. Please file an issue (or let me know) if anyone finds a reason to do this. r? jieyouxu

Rollup merge of #146574 - Zalathar:capture, r=jieyouxu compiletest: Enable new-output-capture by default The new output-capture implementation was added in #146119, but was disabled by default and required opt-in. Since then, I haven't encountered any problems in my own testing/usage, and I haven't heard any problem reports from other contributors who might have opted in. It's unlikely that more opt-in testing will help, so the next step is to enable new-output-capture by default and see if anyone complains. (Hopefully nobody!) If needed, the new default can be overridden (for now) by setting environment variable `COMPILETEST_NEW_OUTPUT_CAPTURE=off`. Please file an issue (or let me know) if anyone finds a reason to do this. r? jieyouxu

compiletest: Enable new-output-capture by default The new output-capture implementation was added in rust-lang/rust#146119, but was disabled by default and required opt-in. Since then, I haven't encountered any problems in my own testing/usage, and I haven't heard any problem reports from other contributors who might have opted in. It's unlikely that more opt-in testing will help, so the next step is to enable new-output-capture by default and see if anyone complains. (Hopefully nobody!) If needed, the new default can be overridden (for now) by setting environment variable `COMPILETEST_NEW_OUTPUT_CAPTURE=off`. Please file an issue (or let me know) if anyone finds a reason to do this. r? jieyouxu

Zalathar added 3 commits September 2, 2025 19:52

Encapsulate output-capture mode in a dedicated enum

ee87a8b

Preliminary plumbing of stdout/stderr as &dyn ConsoleOut

d89a1a0

Replace print statements in runtest with write! or writeln!

5d71a8a

rustbot assigned jieyouxu Sep 2, 2025

This comment was marked as outdated.

Sign in to view

Zalathar force-pushed the capture branch from 8ad7c8d to 9c5e1cb Compare September 2, 2025 11:21

jieyouxu approved these changes Sep 3, 2025

View reviewed changes

src/tools/compiletest/src/lib.rs Show resolved Hide resolved

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 3, 2025

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 4, 2025

Implement compiletest --new-output-capture, in stable Rust

c2ae2e0

Zalathar force-pushed the capture branch from 9c5e1cb to c2ae2e0 Compare September 4, 2025 02:51

jhpratt mentioned this pull request Sep 4, 2025

Rollup of 5 pull requests #146192

Merged

Zalathar mentioned this pull request Sep 4, 2025

Rollup of 5 pull requests #146193

Closed

bors merged commit be1d829 into rust-lang:master Sep 4, 2025
10 checks passed

rustbot added this to the 1.91.0 milestone Sep 4, 2025

Zalathar deleted the capture branch September 4, 2025 13:11

Zalathar mentioned this pull request Sep 15, 2025

compiletest: Enable new-output-capture by default #146574

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

compiletest: Implement an experimental `--new-output-capture` mode #146119

compiletest: Implement an experimental `--new-output-capture` mode #146119

Uh oh!

Zalathar commented Sep 2, 2025 •

edited

Loading

Uh oh!

rustbot commented Sep 2, 2025

Uh oh!

This comment was marked as outdated.

Kobzol commented Sep 2, 2025

Uh oh!

Zalathar commented Sep 2, 2025

Uh oh!

jieyouxu left a comment •

edited by rustbot

Loading

Uh oh!

Uh oh!

jieyouxu commented Sep 3, 2025

Uh oh!

rustbot commented Sep 3, 2025

Uh oh!

jieyouxu commented Sep 4, 2025

Uh oh!

bors commented Sep 4, 2025

Uh oh!

Zalathar commented Sep 4, 2025

Uh oh!

bors commented Sep 4, 2025

Uh oh!

Uh oh!

Zalathar commented Sep 12, 2025

Uh oh!

Uh oh!

compiletest: Implement an experimental --new-output-capture mode #146119

compiletest: Implement an experimental --new-output-capture mode #146119

Uh oh!

Conversation

Zalathar commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Sep 2, 2025

Uh oh!

This comment was marked as outdated.

Kobzol commented Sep 2, 2025

Uh oh!

Zalathar commented Sep 2, 2025

Uh oh!

jieyouxu left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jieyouxu commented Sep 3, 2025

Uh oh!

rustbot commented Sep 3, 2025

Uh oh!

jieyouxu commented Sep 4, 2025

Uh oh!

bors commented Sep 4, 2025

Uh oh!

Zalathar commented Sep 4, 2025

Uh oh!

bors commented Sep 4, 2025

Uh oh!

Uh oh!

Zalathar commented Sep 12, 2025

Uh oh!

Uh oh!

compiletest: Implement an experimental `--new-output-capture` mode #146119

compiletest: Implement an experimental `--new-output-capture` mode #146119

Zalathar commented Sep 2, 2025 •

edited

Loading

jieyouxu left a comment •

edited by rustbot

Loading