bootstrap: allow setting `--jobs` in config.toml #131838

jieyouxu · 2024-10-17T15:06:25Z

Allow setting --jobs in config.toml's [build] section.

[build]
jobs = 0

If this is unset or set to zero in config.toml, we look at --jobs flag. If that is also unset, then we fallback to std::thread::available_parallelism. If that is not available, then we default to 1. The flags and available_parallelism fallback are already setup, this PR just adds a config.toml option to wire that up.

Closes #131836.

r? bootstrap

rustbot · 2024-10-17T15:06:37Z

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

This PR modifies config.example.toml.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

jieyouxu · 2024-10-17T16:08:13Z

@rustbot author

jieyouxu · 2024-10-18T03:07:33Z

@rustbot ready

onur-ozkan · 2024-10-18T04:23:26Z

src/bootstrap/src/core/config/config.rs

+        if let Some(jobs) = jobs {
+            if jobs > 0 {
+                config.jobs = Some(jobs);
+            }
+        }


We should handle it here:

rust/src/bootstrap/src/core/config/config.rs

Line 1293 in e057ca5

config.jobs = Some(threads_from_config(flags.jobs as u32));

Note that there's two layers of precedence, the line there is the top priority --jobs flag. I changed the logic to better reflect the precedence such that:

config.jobs and flags.jobs are initially None.

config.jobs is Some based on --jobs=N if N > 0 and still None otherwise.

If config.jobs is still None, then we fallback to --set build.jobs then [build] jobs. Note that the build toml section is read below this.

I added a precedence test to check this.

(Remark: I found this config layering somewhat hard to follow)

src/bootstrap/src/core/config/tests.rs

config.example.toml

src/bootstrap/src/core/config/config.rs

jieyouxu · 2024-10-18T06:22:34Z

@rustbot author

jieyouxu · 2024-10-18T10:32:33Z

@rustbot ready

jieyouxu · 2024-10-19T05:22:37Z

@rustbot ready

onur-ozkan

r=me with/without the nit suggestion

src/bootstrap/src/core/config/config.rs

onur-ozkan · 2024-10-19T05:41:14Z

Thanks!

@bors r+

bors · 2024-10-19T05:41:17Z

📌 Commit 65458ae has been approved by onur-ozkan

It is now in the queue for this repository.

bootstrap: allow setting `--jobs` in config.toml Allow setting `--jobs` in config.toml's `[build]` section. ```toml [build] jobs = 0 ``` If this is unset or set to zero in config.toml, we look at `--jobs` flag. If that is also unset, then we fallback to `std::thread::available_parallelism`. If that is not available, then we default to `1`. The flags and `available_parallelism` fallback are already setup, this PR just adds a config.toml option to wire that up. Closes rust-lang#131836. r? bootstrap

…kingjubilee Rollup of 8 pull requests Successful merges: - rust-lang#127462 (std: uefi: Add basic Env variables) - rust-lang#131537 (Fix range misleading field access) - rust-lang#131838 (bootstrap: allow setting `--jobs` in config.toml) - rust-lang#131871 (x86-32 float return for 'Rust' ABI: treat all float types consistently) - rust-lang#131876 (compiler: Use LLVM's Comdat support) - rust-lang#131890 (Update `use` keyword docs to describe precise capturing) - rust-lang#131899 (Mark unexpected variant res suggestion as having placeholders) - rust-lang#131908 (rustdoc: Switch from FxHash to sha256 for static file hashing.) r? `@ghost` `@rustbot` modify labels: rollup

bootstrap: allow setting `--jobs` in config.toml Allow setting `--jobs` in config.toml's `[build]` section. ```toml [build] jobs = 0 ``` If this is unset or set to zero in config.toml, we look at `--jobs` flag. If that is also unset, then we fallback to `std::thread::available_parallelism`. If that is not available, then we default to `1`. The flags and `available_parallelism` fallback are already setup, this PR just adds a config.toml option to wire that up. Closes rust-lang#131836. r? bootstrap

…iaskrgr Rollup of 10 pull requests Successful merges: - rust-lang#127462 (std: uefi: Add basic Env variables) - rust-lang#131537 (Fix range misleading field access) - rust-lang#131838 (bootstrap: allow setting `--jobs` in config.toml) - rust-lang#131871 (x86-32 float return for 'Rust' ABI: treat all float types consistently) - rust-lang#131876 (compiler: Use LLVM's Comdat support) - rust-lang#131890 (Update `use` keyword docs to describe precise capturing) - rust-lang#131899 (Mark unexpected variant res suggestion as having placeholders) - rust-lang#131908 (rustdoc: Switch from FxHash to sha256 for static file hashing.) - rust-lang#131916 (small interpreter error cleanup) - rust-lang#131919 (zero-sized accesses are fine on null pointers) r? `@ghost` `@rustbot` modify labels: rollup

bootstrap: allow setting `--jobs` in config.toml Allow setting `--jobs` in config.toml's `[build]` section. ```toml [build] jobs = 0 ``` If this is unset or set to zero in config.toml, we look at `--jobs` flag. If that is also unset, then we fallback to `std::thread::available_parallelism`. If that is not available, then we default to `1`. The flags and `available_parallelism` fallback are already setup, this PR just adds a config.toml option to wire that up. Closes rust-lang#131836. r? bootstrap

…iaskrgr Rollup of 9 pull requests Successful merges: - rust-lang#127462 (std: uefi: Add basic Env variables) - rust-lang#131537 (Fix range misleading field access) - rust-lang#131838 (bootstrap: allow setting `--jobs` in config.toml) - rust-lang#131871 (x86-32 float return for 'Rust' ABI: treat all float types consistently) - rust-lang#131890 (Update `use` keyword docs to describe precise capturing) - rust-lang#131899 (Mark unexpected variant res suggestion as having placeholders) - rust-lang#131908 (rustdoc: Switch from FxHash to sha256 for static file hashing.) - rust-lang#131916 (small interpreter error cleanup) - rust-lang#131919 (zero-sized accesses are fine on null pointers) r? `@ghost` `@rustbot` modify labels: rollup

…iaskrgr Rollup of 8 pull requests Successful merges: - rust-lang#127462 (std: uefi: Add basic Env variables) - rust-lang#131537 (Fix range misleading field access) - rust-lang#131838 (bootstrap: allow setting `--jobs` in config.toml) - rust-lang#131890 (Update `use` keyword docs to describe precise capturing) - rust-lang#131899 (Mark unexpected variant res suggestion as having placeholders) - rust-lang#131908 (rustdoc: Switch from FxHash to sha256 for static file hashing.) - rust-lang#131916 (small interpreter error cleanup) - rust-lang#131919 (zero-sized accesses are fine on null pointers) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of rust-lang#131838 - jieyouxu:boopjob, r=onur-ozkan bootstrap: allow setting `--jobs` in config.toml Allow setting `--jobs` in config.toml's `[build]` section. ```toml [build] jobs = 0 ``` If this is unset or set to zero in config.toml, we look at `--jobs` flag. If that is also unset, then we fallback to `std::thread::available_parallelism`. If that is not available, then we default to `1`. The flags and `available_parallelism` fallback are already setup, this PR just adds a config.toml option to wire that up. Closes rust-lang#131836. r? bootstrap

rustbot assigned onur-ozkan Oct 17, 2024

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Oct 17, 2024

jieyouxu force-pushed the boopjob branch from 49fcc6b to 6a6d20f Compare October 17, 2024 15:07

jieyouxu mentioned this pull request Oct 17, 2024

Bootstrap feature request: set --jobs in config.toml #131836

Closed

This comment has been minimized.

Sign in to view

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 17, 2024

jieyouxu force-pushed the boopjob branch 2 times, most recently from 8302cba to e057ca5 Compare October 18, 2024 02:54

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 18, 2024

onur-ozkan reviewed Oct 18, 2024

View reviewed changes

config.example.toml Show resolved Hide resolved

jieyouxu force-pushed the boopjob branch 2 times, most recently from 4adaf51 to 2fdc3b1 Compare October 18, 2024 05:25

jieyouxu commented Oct 18, 2024

View reviewed changes

src/bootstrap/src/core/config/config.rs Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 18, 2024

jieyouxu force-pushed the boopjob branch from 2fdc3b1 to a18e389 Compare October 18, 2024 08:38

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 18, 2024

jieyouxu added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 18, 2024

jieyouxu force-pushed the boopjob branch from a18e389 to 667d659 Compare October 19, 2024 04:03

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 19, 2024

onur-ozkan approved these changes Oct 19, 2024

View reviewed changes

src/bootstrap/src/core/config/config.rs Outdated Show resolved Hide resolved

bootstrap: allow setting --jobs in config.toml

65458ae

jieyouxu force-pushed the boopjob branch from 667d659 to 65458ae Compare October 19, 2024 05:36

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 19, 2024

workingjubilee mentioned this pull request Oct 19, 2024

Rollup of 8 pull requests #131914

Closed

matthiaskrgr mentioned this pull request Oct 19, 2024

Rollup of 10 pull requests #131922

Closed

matthiaskrgr mentioned this pull request Oct 19, 2024

Rollup of 9 pull requests #131924

Closed

matthiaskrgr mentioned this pull request Oct 19, 2024

Rollup of 8 pull requests #131934

Merged

bors merged commit 21475b7 into rust-lang:master Oct 19, 2024
6 checks passed

rustbot added this to the 1.84.0 milestone Oct 19, 2024

jieyouxu deleted the boopjob branch October 21, 2024 08:42

bootstrap: allow setting --jobs in config.toml #131838

bootstrap: allow setting --jobs in config.toml #131838

Uh oh!

Conversation

jieyouxu commented Oct 17, 2024

Uh oh!

rustbot commented Oct 17, 2024

Uh oh!

This comment has been minimized.

jieyouxu commented Oct 17, 2024

Uh oh!

jieyouxu commented Oct 18, 2024

Uh oh!

onur-ozkan Oct 18, 2024

Choose a reason for hiding this comment

Uh oh!

jieyouxu Oct 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jieyouxu Oct 18, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

jieyouxu commented Oct 18, 2024

Uh oh!

jieyouxu commented Oct 18, 2024

Uh oh!

jieyouxu commented Oct 19, 2024

Uh oh!

onur-ozkan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

onur-ozkan commented Oct 19, 2024

Uh oh!

bors commented Oct 19, 2024

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

bootstrap: allow setting `--jobs` in config.toml #131838

bootstrap: allow setting `--jobs` in config.toml #131838

jieyouxu Oct 18, 2024 •

edited

Loading