bootstrap.py: Suggest disabling download-ci-llvm option if url fails to download #93885

Badel2 · 2022-02-11T01:50:32Z

I got an error when trying to build the compiler using an old commit, and it turns out it was because the option download-ci-llvm was implicitly set to true. So this pull request tries to add a help message for other people that may run into the same problem.

To reproduce my error:

git checkout 8d7707f3c4f72e6eb334d897354beca692b265d1
./x.py test
[...]
spurious failure, trying again
downloading https://ci-artifacts.rust-lang.org/rustc-builds/db002a06ae9154a35d410550bc5132df883d7baa/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
curl: (22) The requested URL returned error: 404 

failed to run: curl -# -y 30 -Y 10 --connect-timeout 30 --retry 3 -Sf -o /tmp/tmp8g13rb4n https://ci-artifacts.rust-lang.org/rustc-builds/db002a06ae9154a35d410550bc5132df883d7baa/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
Build completed unsuccessfully in 0:00:46

This is my config.toml:

# Includes one of the default files in src/bootstrap/defaults
profile = "compiler"
changelog-seen = 2

[rust]
debug = true

To reproduce an error with this branch:

Change line 618 of bootstrap.py to

        url = "rustc-builds-error404/{}".format(llvm_sha)

Delete llvm and cached tarball, and set llvm.download-ci-llvm=true in config.toml.

./x.py test
[...]
spurious failure, trying again
downloading https://ci-artifacts.rust-lang.org/rustc-builds-error404/719b04ca99be0c78e09a8ec5e2eda082a5d8ccae/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
curl: (22) The requested URL returned error: 404 

failed to run: curl -# -y 30 -Y 10 --connect-timeout 30 --retry 3 -Sf -o /tmp/tmpesl1ydvo https://ci-artifacts.rust-lang.org/rustc-builds-error404/719b04ca99be0c78e09a8ec5e2eda082a5d8ccae/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
error: failed to download llvm from ci
help: old builds get deleted after a certain time
help: if trying to compile an old commit of rustc, disable `download-ci-llvm` in config.toml:

[llvm]
download-ci-llvm = false

Build completed unsuccessfully in 0:00:01

Regarding the implementation, I expected to be able to use a try/catch block in _download_ci_llvm, but the run function calls sys.exit instead of raising an exception so that's not possible. Also, suggestions for better wording of the help message are welcome.

rust-highfive · 2022-02-11T01:50:34Z

r? @Mark-Simulacrum

(rust-highfive has picked a reviewer for you, use r? to override)

Mark-Simulacrum · 2022-02-11T23:18:37Z

@bors r+ rollup

This seems pretty positive to me.

bors · 2022-02-11T23:18:38Z

📌 Commit dfa9d64 has been approved by Mark-Simulacrum

…rk-Simulacrum bootstrap.py: Suggest disabling download-ci-llvm option if url fails to download I got an error when trying to build the compiler using an old commit, and it turns out it was because the option `download-ci-llvm` was implicitly set to true. So this pull request tries to add a help message for other people that may run into the same problem. To reproduce my error: ``` git checkout 8d7707f ./x.py test [...] spurious failure, trying again downloading https://ci-artifacts.rust-lang.org/rustc-builds/db002a06ae9154a35d410550bc5132df883d7baa/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz curl: (22) The requested URL returned error: 404 failed to run: curl -# -y 30 -Y 10 --connect-timeout 30 --retry 3 -Sf -o /tmp/tmp8g13rb4n https://ci-artifacts.rust-lang.org/rustc-builds/db002a06ae9154a35d410550bc5132df883d7baa/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz Build completed unsuccessfully in 0:00:46 ``` This is my `config.toml`: ``` # Includes one of the default files in src/bootstrap/defaults profile = "compiler" changelog-seen = 2 [rust] debug = true ``` To reproduce an error with this branch: Change line 618 of bootstrap.py to ``` url = "rustc-builds-error404/{}".format(llvm_sha) ``` Delete llvm and cached tarball, and set `llvm.download-ci-llvm=true` in config.toml. ``` ./x.py test [...] spurious failure, trying again downloading https://ci-artifacts.rust-lang.org/rustc-builds-error404/719b04ca99be0c78e09a8ec5e2eda082a5d8ccae/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz curl: (22) The requested URL returned error: 404 failed to run: curl -# -y 30 -Y 10 --connect-timeout 30 --retry 3 -Sf -o /tmp/tmpesl1ydvo https://ci-artifacts.rust-lang.org/rustc-builds-error404/719b04ca99be0c78e09a8ec5e2eda082a5d8ccae/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz error: failed to download llvm from ci help: old builds get deleted after a certain time help: if trying to compile an old commit of rustc, disable `download-ci-llvm` in config.toml: [llvm] download-ci-llvm = false Build completed unsuccessfully in 0:00:01 ``` Regarding the implementation, I expected to be able to use a try/catch block in `_download_ci_llvm`, but the `run` function calls `sys.exit` instead of raising an exception so that's not possible. Also, suggestions for better wording of the help message are welcome.

hkratz · 2022-02-12T05:58:52Z

src/bootstrap/bootstrap.py

@@ -178,6 +179,8 @@ def run(args, verbose=False, exception=False, is_bootstrap=False, **kwargs):
    code = ret.wait()
    if code != 0:
        err = "failed to run: " + ' '.join(args)
+        if help_on_error is not None:
+            err = f"{err}\n{help_on_error}"


Unfortunately we cannot use f-strings, which are Python 3.6+. x.py is currently required to be compatible with Python 2.x though there are discussions about bumping it to Python 3 (Zulip thread).

This caused the rollup failure in #93929.

Fixed, thank you.

matthiaskrgr · 2022-02-12T06:03:22Z

@bors r-

Mark-Simulacrum · 2022-02-12T13:57:21Z

@bors r+

bors · 2022-02-12T13:57:22Z

📌 Commit 410145e has been approved by Mark-Simulacrum

…rk-Simulacrum bootstrap.py: Suggest disabling download-ci-llvm option if url fails to download I got an error when trying to build the compiler using an old commit, and it turns out it was because the option `download-ci-llvm` was implicitly set to true. So this pull request tries to add a help message for other people that may run into the same problem. To reproduce my error: ``` git checkout 8d7707f ./x.py test [...] spurious failure, trying again downloading https://ci-artifacts.rust-lang.org/rustc-builds/db002a06ae9154a35d410550bc5132df883d7baa/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz curl: (22) The requested URL returned error: 404 failed to run: curl -# -y 30 -Y 10 --connect-timeout 30 --retry 3 -Sf -o /tmp/tmp8g13rb4n https://ci-artifacts.rust-lang.org/rustc-builds/db002a06ae9154a35d410550bc5132df883d7baa/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz Build completed unsuccessfully in 0:00:46 ``` This is my `config.toml`: ``` # Includes one of the default files in src/bootstrap/defaults profile = "compiler" changelog-seen = 2 [rust] debug = true ``` To reproduce an error with this branch: Change line 618 of bootstrap.py to ``` url = "rustc-builds-error404/{}".format(llvm_sha) ``` Delete llvm and cached tarball, and set `llvm.download-ci-llvm=true` in config.toml. ``` ./x.py test [...] spurious failure, trying again downloading https://ci-artifacts.rust-lang.org/rustc-builds-error404/719b04ca99be0c78e09a8ec5e2eda082a5d8ccae/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz curl: (22) The requested URL returned error: 404 failed to run: curl -# -y 30 -Y 10 --connect-timeout 30 --retry 3 -Sf -o /tmp/tmpesl1ydvo https://ci-artifacts.rust-lang.org/rustc-builds-error404/719b04ca99be0c78e09a8ec5e2eda082a5d8ccae/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz error: failed to download llvm from ci help: old builds get deleted after a certain time help: if trying to compile an old commit of rustc, disable `download-ci-llvm` in config.toml: [llvm] download-ci-llvm = false Build completed unsuccessfully in 0:00:01 ``` Regarding the implementation, I expected to be able to use a try/catch block in `_download_ci_llvm`, but the `run` function calls `sys.exit` instead of raising an exception so that's not possible. Also, suggestions for better wording of the help message are welcome.

…askrgr Rollup of 9 pull requests Successful merges: - rust-lang#89926 (make `Instant::{duration_since, elapsed, sub}` saturating and remove workarounds) - rust-lang#90532 (More informative error message for E0015) - rust-lang#93810 (Improve chalk integration) - rust-lang#93851 (More practical examples for `Option::and_then` & `Result::and_then`) - rust-lang#93885 (bootstrap.py: Suggest disabling download-ci-llvm option if url fails to download) - rust-lang#93886 (Stabilise inherent_ascii_escape (FCP in rust-lang#77174)) - rust-lang#93930 (add link to format_args! when mention it in docs) - rust-lang#93936 (Couple of driver cleanups) - rust-lang#93944 (Don't relabel to a team if there is already a team label) Failed merges: r? `@ghost` `@rustbot` modify labels: rollup

rust-highfive assigned Mark-Simulacrum Feb 11, 2022

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Feb 11, 2022

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 11, 2022

matthiaskrgr mentioned this pull request Feb 12, 2022

Rollup of 8 pull requests #93929

Closed

hkratz reviewed Feb 12, 2022

View reviewed changes

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Feb 12, 2022

Suggest disabling download-ci-llvm option if url fails to download

410145e

Badel2 force-pushed the error-download-ci-llvm branch from dfa9d64 to 410145e Compare February 12, 2022 13:23

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Feb 12, 2022

matthiaskrgr mentioned this pull request Feb 12, 2022

Rollup of 5 pull requests #93943

Closed

matthiaskrgr mentioned this pull request Feb 13, 2022

Rollup of 9 pull requests #93956

Merged

bors merged commit b90a369 into rust-lang:master Feb 13, 2022

rustbot added this to the 1.60.0 milestone Feb 13, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

bootstrap.py: Suggest disabling download-ci-llvm option if url fails to download #93885

bootstrap.py: Suggest disabling download-ci-llvm option if url fails to download #93885

Uh oh!

Badel2 commented Feb 11, 2022

Uh oh!

rust-highfive commented Feb 11, 2022

Uh oh!

Mark-Simulacrum commented Feb 11, 2022

Uh oh!

bors commented Feb 11, 2022

Uh oh!

hkratz Feb 12, 2022

Uh oh!

Badel2 Feb 12, 2022

Uh oh!

matthiaskrgr commented Feb 12, 2022

Uh oh!

Mark-Simulacrum commented Feb 12, 2022

Uh oh!

bors commented Feb 12, 2022

Uh oh!

Uh oh!

bootstrap.py: Suggest disabling download-ci-llvm option if url fails to download #93885

bootstrap.py: Suggest disabling download-ci-llvm option if url fails to download #93885

Uh oh!

Conversation

Badel2 commented Feb 11, 2022

Uh oh!

rust-highfive commented Feb 11, 2022

Uh oh!

Mark-Simulacrum commented Feb 11, 2022

Uh oh!

bors commented Feb 11, 2022

Uh oh!

hkratz Feb 12, 2022

Choose a reason for hiding this comment

Uh oh!

Badel2 Feb 12, 2022

Choose a reason for hiding this comment

Uh oh!

matthiaskrgr commented Feb 12, 2022

Uh oh!

Mark-Simulacrum commented Feb 12, 2022

Uh oh!

bors commented Feb 12, 2022

Uh oh!

Uh oh!