Skip to content

Consolidate stage directories and group logs in bootstrap #145295

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Aug 13, 2025

Conversation

Kobzol
Copy link
Member

@Kobzol Kobzol commented Aug 12, 2025

My post-stage-0-redesign bootstrap fixes aren't done yet, but I think that enough steps have been migrated to the new system that it makes sense to actually modify the directories on disk, and what gets printed when bootstrap runs, so that it actually corresponds to the new system. Before, the printed stages didn't always make sense.

This PR:

  • Fixes the numbering of stageN directories in the build directory. It was not corresponding to the correct stages before; notice that I did not modify bootstrap/README.md, as it was essentially describing what happens after this PR (first commit). This means that stageN-<foo> now actually corresponds to <foo> (e.g. rustc or tools) that is stage N. Before in some cases it means that it was built with the stage N compiler. So for example stage0-rustc became stage1-rustc.
  • Unifies all steps that output a build group to use the Builder::msg method. It's probably not the final stage, and some of the test steps might not be fully accurate yet, because I didn't fix test step numbering yet, but I think that it's a clear improvement from before, and now that everything uses the same method, we can easily make changes across the board, to ensure that it stays unified (second commit).

r? @jieyouxu

try-job: dist-x86_64-msvc
try-job: dist-x86_64-linux

@rustbot rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Aug 12, 2025
Copy link
Member

@jieyouxu jieyouxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, tiny nits, r=me after.

@jieyouxu
Copy link
Member

@rustbot author

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 12, 2025
@Kobzol
Copy link
Member Author

Kobzol commented Aug 12, 2025

Resolved review remarks. I would bump the priority of this, because it will be very conflict-prone with anything done in bootstrap.

Copy link
Member

@jieyouxu jieyouxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@jieyouxu
Copy link
Member

@bors r+ rollup=5 (conflicts with other bootstrap work, conflict-prone)

@bors
Copy link
Collaborator

bors commented Aug 12, 2025

📌 Commit f7defc0 has been approved by jieyouxu

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 12, 2025
@jieyouxu
Copy link
Member

Aaa
@bors p=5 rollup=never

bors added a commit that referenced this pull request Aug 12, 2025
Consolidate stage directories and group logs in bootstrap

My post-stage-0-redesign bootstrap fixes aren't done yet, but I think that enough steps have been migrated to the new system that it makes sense to actually modify the directories on disk, and what gets printed when bootstrap runs, so that it actually corresponds to the new system. Before, the printed stages didn't always make sense.

This PR:
- Fixes the numbering of `stageN` directories in the build directory. It was not corresponding to the correct stages before; notice that I did not modify `bootstrap/README.md`, as it was essentially describing what happens after this PR (first commit).
- Unifies all steps that output a build group to use the `Builder::msg` method. It's probably not the final stage, and some of the test steps might not be fully accurate yet, because I didn't fix test step numbering yet, but I think that it's a clear improvement from before, and now that everything uses the same method, we can easily make changes across the board, to ensure that it stays unified (second commit).

r? `@jieyouxu`
@bors
Copy link
Collaborator

bors commented Aug 12, 2025

⌛ Testing commit f7defc0 with merge 5e2003d...

@bors
Copy link
Collaborator

bors commented Aug 12, 2025

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 12, 2025
@rust-log-analyzer

This comment has been minimized.

@rustbot rustbot added A-CI Area: Our Github Actions CI A-rustc-dev-guide Area: rustc-dev-guide labels Aug 12, 2025
@rustbot rustbot added the T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. label Aug 12, 2025
@rustbot
Copy link
Collaborator

rustbot commented Aug 12, 2025

The rustc-dev-guide subtree was changed. If this PR only touches the dev guide consider submitting a PR directly to rust-lang/rustc-dev-guide otherwise thank you for updating the dev guide with your changes.

cc @BoxyUwU, @jieyouxu, @Kobzol, @tshepang

@Kobzol
Copy link
Member Author

Kobzol commented Aug 12, 2025

@bors try

@rust-bors

This comment has been minimized.

rust-bors bot added a commit that referenced this pull request Aug 12, 2025
Consolidate stage directories and group logs in bootstrap

try-job: dist-x86_64-msvc
try-job: dist-x86_64-linux
@rust-bors
Copy link

rust-bors bot commented Aug 12, 2025

☀️ Try build successful (CI)
Build commit: 6f046d0 (6f046d0222ccb364b63d9ce259d40e6494353539, parent: d9dba3a55476ae2da5d4e5bce8a81b341c675750)

@Kobzol
Copy link
Member Author

Kobzol commented Aug 12, 2025

Fixes CI failure.

@bors r=jieyouxu

@bors
Copy link
Collaborator

bors commented Aug 12, 2025

📌 Commit 4360c52 has been approved by jieyouxu

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 12, 2025
@bors
Copy link
Collaborator

bors commented Aug 12, 2025

⌛ Testing commit 4360c52 with merge 1553adf...

@bors
Copy link
Collaborator

bors commented Aug 13, 2025

☀️ Test successful - checks-actions
Approved by: jieyouxu
Pushing 1553adf to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Aug 13, 2025
@bors bors merged commit 1553adf into rust-lang:master Aug 13, 2025
12 checks passed
@rustbot rustbot added this to the 1.91.0 milestone Aug 13, 2025
Copy link
Contributor

What is this? This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing 8e62bfd (parent) -> 1553adf (this PR)

Test differences

Show 4 test diffs

4 doctest diffs were found. These are ignored, as they are noisy.

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard 1553adfe6884a8f6c28f5a673d3e605535ee0113 --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

  1. dist-x86_64-apple: 12716.7s -> 7941.2s (-37.6%)
  2. x86_64-apple-2: 5341.4s -> 6264.9s (17.3%)
  3. aarch64-apple: 5000.5s -> 5830.3s (16.6%)
  4. x86_64-gnu-llvm-19-2: 6275.1s -> 7190.7s (14.6%)
  5. x86_64-apple-1: 10604.1s -> 9291.6s (-12.4%)
  6. dist-aarch64-msvc: 5250.7s -> 5847.6s (11.4%)
  7. dist-i586-gnu-i586-i686-musl: 6213.9s -> 5522.7s (-11.1%)
  8. dist-aarch64-linux: 6238.5s -> 5632.9s (-9.7%)
  9. aarch64-msvc-2: 4761.0s -> 5199.9s (9.2%)
  10. arm-android: 6159.5s -> 6648.4s (7.9%)
How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (1553adf): comparison URL.

Overall result: ❌✅ regressions and improvements - no action needed

@rustbot label: -perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
0.1% [0.0%, 0.1%] 2
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-0.3% [-0.5%, -0.1%] 2
All ❌✅ (primary) - - 0

Max RSS (memory usage)

Results (primary 3.6%, secondary -3.8%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
3.6% [3.6%, 3.6%] 1
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-3.8% [-5.2%, -1.5%] 4
All ❌✅ (primary) 3.6% [3.6%, 3.6%] 1

Cycles

Results (secondary -1.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
6.0% [6.0%, 6.0%] 1
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-3.5% [-4.1%, -2.9%] 3
All ❌✅ (primary) - - 0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 465.671s -> 464.093s (-0.34%)
Artifact size: 377.43 MiB -> 377.76 MiB (0.09%)

@Kobzol Kobzol deleted the unify-stages branch August 13, 2025 04:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-CI Area: Our Github Actions CI A-rustc-dev-guide Area: rustc-dev-guide A-testsuite Area: The testsuite used to check the correctness of rustc merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants