Distribute cg_clif as rustup component on the nightly channel #81746

bjorn3 · 2021-02-04T10:26:21Z

This makes it possible to use cg_clif using:

$ rustup component add rustc-codegen-cranelift-preview --toolchain nightly
$ RUSTFLAGS="-Zcodegen-backend=cranelift" cargo +nightly build

cc rust-lang/compiler-team#405.
r? @Mark-Simulacrum

bjorn3 · 2021-02-08T08:33:11Z

./x.py build seems to build another rustc stage now. Why does that happen?

src/bootstrap/dist.rs

Mark-Simulacrum · 2021-02-12T02:41:59Z

For the bootstrap tests, I'd recommend an x.py build locally that tries to reproduce it (--dry-run should help with getting quick results), and running with -vvv to see what is invoking the newly added rustc builds. I suspect it's something in the codegenbackend step in compile.rs, but haven't investigated myself (and would prefer not to).

bjorn3 · 2021-02-12T11:00:47Z

--dry-run should help with getting quick results

Thanks for the tip!

I got this with download-ci-llvm:

$ ./x.py build --dry-run -v
[...]
running: /home/bjorn/Documenten/rust/build/bootstrap/debug/bootstrap build --dry-run -v
thread 'main' panicked at 'std::fs::read_to_string(ci_llvm.join("link-type.txt")) failed with No such file or directory (os error 2)', src/bootstrap/config.rs:823:33
stack backtrace:
   0: rust_begin_unwind
             at /rustc/05b6023675d77979637b04a350c85903fbf59257/library/std/src/panicking.rs:493:5
   1: std::panicking::begin_panic_fmt
             at /rustc/05b6023675d77979637b04a350c85903fbf59257/library/std/src/panicking.rs:435:5
   2: bootstrap::config::Config::parse
             at ./src/bootstrap/config.rs:823:33
   3: bootstrap::main
             at ./src/bootstrap/bin/main.rs:14:18

Disabling it fixes the problem.

bjorn3 · 2021-02-12T13:05:44Z

Previously it would build assemble stage 1 -> (rustc stage 0 and codegen backend stage 0) and std stage 1. Now it also builds codegen backend stage 1 -> rustc stage 1. Removing the DEFAULT override in the Step implementation fixed it.

bjorn3 · 2021-02-12T13:51:01Z

Let's check if the rustup component is correctly generated.

@bors try

bors · 2021-02-12T13:51:17Z

⌛ Trying commit 909dbc1857b49279156b64061c13ab55c9f03b80 with merge 4bbf0fa74cbc11268c4f6f9d641e12b360065fc1...

bors · 2021-02-12T14:40:23Z

☀️ Try build successful - checks-actions
Build commit: 4bbf0fa74cbc11268c4f6f9d641e12b360065fc1 (4bbf0fa74cbc11268c4f6f9d641e12b360065fc1)

bjorn3 · 2021-02-12T15:09:29Z

I must have made a mistake somewhere. rustup-toolchain-install-master 4bbf0fa74cbc11268c4f6f9d641e12b360065fc1 -c rustc-codegen-cranelift fails, while -c miri and -c rust-analyzer work.

bjorn3 · 2021-02-13T18:37:20Z

I think I should wire up compiler/rustc_codegen_cranelift to the tidy license checking. It isn't currently checked due to being in a different workspace as far as I know.

bjorn3 · 2021-02-19T17:07:47Z

@bors try

bors · 2021-02-19T17:07:58Z

⌛ Trying commit de705b10023b1ed5b9ad92810e3a0ea1dc29caee with merge 874d9737dc9cca284c21e5b4aaf40bf57502947f...

…rk-Simulacrum Distribute cg_clif as rustup component on the nightly channel This makes it possible to use cg_clif using: ```bash $ rustup component add rustc-codegen-cranelift-preview --toolchain nightly $ RUSTFLAGS="-Zcodegen-backend=cranelift" cargo +nightly build ``` cc rust-lang/compiler-team#405. r? `@Mark-Simulacrum`

pass `CODEGEN_BACKENDS` to docker The backends to build are now defined in the `CODEGEN_BACKENDS` env var. It's correctly set in CI, but wasn't passed to docker, hence cg_clif wasn't actually built in rust-lang#81746. r? `@Kobzol:` I locally tried `CODEGEN_BACKENDS="cranelift" DEPLOY=1 src/ci/docker/run.sh dist-x86_64-linux` and this change was enough for `ci/run.sh` to read the env var. So I'll try as-is and we'll see.

tgross35 · 2023-10-30T17:58:05Z

I can't be the only one with this question - how does one actually put this to use after rustup component add rustc-codegen-cranelift-x86_64-unknown-linux-gnu?

Edit: I thought #117328 landed by rustc 1.75.0-nightly (608e9682f 2023-10-29) but I guess not. Once it does, seems it will just work by passing rustc -Zcodegen-backend=??

Skgland · 2023-10-30T19:00:21Z

I can't be the only one with this question - how does one actually put this to use after rustup component add rustc-codegen-cranelift-x86_64-unknown-linux-gnu?

That looks like you are mixing the cranelift codegen backend component with the compilation target toolchain.

Edit: I thought #117328 landed by rustc 1.75.0-nightly (608e9682f 2023-10-29) but I guess not. Once it does, seems it will just work by passing rustc -Zcodegen-backend=??

As github sais this was merged two days ago I would have expected this to be in the current nightly and based on the PR description I would expect the following to work

depending on whether nightly is installed
a. rustup toolchain install nightly
b. rustup toolchain update
rustup component add rustc-codegen-cranelift-preview --toolchain nightly
RUSTFLAGS="-Zcodegen-backend=cranelift" cargo +nightly build

1b and 2 succeeded the later suprisingly quickly, but 3 failed with

$ RUSTFLAGS="-Zcodegen-backend=cranelift" cargo +nightly build
error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `/home/bennet/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/rustc - --crate-name ___ --print=file-names -Zcodegen-backend=cranelift --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=split-debuginfo --print=crate-name --print=cfg` (exit status: 1)
  --- stderr
  error: failed to find a `codegen-backends` folder in the sysroot candidates:
         * /home/bennet/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu
         * /home/bennet/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu

Kobzol · 2023-10-30T19:02:42Z

The build of the cranelift component wasn't working in this PR. It was fixed in #117328, so it should hopefully work in tomorrow's nightly.

tgross35 · 2023-10-30T19:04:49Z

I can't be the only one with this question - how does one actually put this to use after rustup component add rustc-codegen-cranelift-x86_64-unknown-linux-gnu?

That looks like you are mixing the cranelift codegen backend component with the compilation target toolchain.

I think rustc-codegen-cranelift-preview is just a shortcut to the component rustc-codegen-cranelift-x86_64-unknown-linux-gnu (host target as applicable), I just didn't know about it since it doesn't show up in rustup component list

Edit: I thought #117328 landed by rustc 1.75.0-nightly (608e9682f 2023-10-29) but I guess not. Once it does, seems it will just work by passing rustc -Zcodegen-backend=??

As github sais this was merged two days ago I would have expected this to be in the current nightly and based on the PR description I would expect the following to work

Opened two days ago but merged 15 hours ago

g2p · 2023-10-30T21:30:23Z

~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/manifest-rustc-codegen-cranelift-preview-x86_64-unknown-linux-gnu is just a handful of documentation files, until the pipeline is entirely working.

tgross35 · 2023-10-31T17:16:07Z

2023-10-30 nightly indeed works! rustc -Zcodegen-backend=cranelift / RUSTFLAGS=-Zcodegen-backend=cranelift cargo build

lqd · 2023-10-31T17:23:29Z

@tgross35 https://bjorn3.github.io/2023/10/31/progress-report-oct-2023.html also describes the couple of other ways you can use cargo's support for selecting the backend to use.

Now that cranelift is distributed with Rustup, I find myself constantly searching for the rust-lang/rust#81746 PR. I think that it would be useful to also describe the Rustup installation/usage instructions here in README.

…r=albertlarsan68 Distribute rustc_codegen_cranelift for Windows With support for raw-dylib recently added to cg_clif, and inline assembly support working on Windows for quite a while now, all blockers for distributing cg_clif on Windows that I mentioned in rust-lang#81746 (comment) are fixed now.

…albertlarsan68 Distribute rustc_codegen_cranelift for Windows With support for raw-dylib recently added to cg_clif, and inline assembly support working on Windows for quite a while now, all blockers for distributing cg_clif on Windows that I mentioned in rust-lang#81746 (comment) are fixed now.

…san68 Distribute rustc_codegen_cranelift for Windows With support for raw-dylib recently added to cg_clif, and inline assembly support working on Windows for quite a while now, all blockers for distributing cg_clif on Windows that I mentioned in rust-lang/rust#81746 (comment) are fixed now.

bjorn3 added T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. A-cranelift Things relevant to the [future] cranelift backend labels Feb 4, 2021

rust-highfive assigned Mark-Simulacrum Feb 4, 2021

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Feb 4, 2021

This comment has been minimized.

Sign in to view

bjorn3 force-pushed the cg_clif_rustup_component branch from 077c3ee to b64122f Compare February 4, 2021 10:54

This comment has been minimized.

Sign in to view

bjorn3 force-pushed the cg_clif_rustup_component branch from b64122f to 06d3150 Compare February 4, 2021 11:12

This comment has been minimized.

Sign in to view

bjorn3 force-pushed the cg_clif_rustup_component branch from 8b5ccb9 to c341ecf Compare February 8, 2021 08:54

Mark-Simulacrum reviewed Feb 12, 2021

View reviewed changes

src/bootstrap/dist.rs Outdated Show resolved Hide resolved

Mark-Simulacrum added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 12, 2021

bjorn3 force-pushed the cg_clif_rustup_component branch from c341ecf to cd173e6 Compare February 12, 2021 13:05

bjorn3 mentioned this pull request Feb 13, 2021

Add a profile option to select the codegen backend rust-lang/cargo#9118

Merged

oxalica mentioned this pull request Oct 30, 2023

Add rustc-codegen-cranelift component to rust-overlay for nightly oxalica/rust-overlay#142

Closed

github-actions bot mentioned this pull request Oct 31, 2023

Automated pull from upstream master ferrocene/ferrocene#74

Merged

lqd mentioned this pull request Oct 31, 2023

CI: Bootstrap no longer documents compiler crates #117430

Closed

Kobzol mentioned this pull request Nov 9, 2023

Add Rustup installation instructions to README rust-lang/rustc_codegen_cranelift#1421

Merged

rami3l mentioned this pull request Apr 5, 2024

Distribute GCC backend rust-lang/rustup#3756

Open

bjorn3 mentioned this pull request Aug 16, 2024

Distribute rustc_codegen_cranelift for Windows #128939

Merged

Distribute cg_clif as rustup component on the nightly channel #81746

Distribute cg_clif as rustup component on the nightly channel #81746

Conversation

bjorn3 commented Feb 4, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

bjorn3 commented Feb 8, 2021

Uh oh!

This comment has been minimized.

Uh oh!

Mark-Simulacrum commented Feb 12, 2021

Uh oh!

bjorn3 commented Feb 12, 2021

Uh oh!

bjorn3 commented Feb 12, 2021

Uh oh!

bjorn3 commented Feb 12, 2021

Uh oh!

bors commented Feb 12, 2021

Uh oh!

bors commented Feb 12, 2021

Uh oh!

bjorn3 commented Feb 12, 2021

Uh oh!

bjorn3 commented Feb 13, 2021

Uh oh!

bjorn3 commented Feb 19, 2021

Uh oh!

bors commented Feb 19, 2021

Uh oh!

tgross35 commented Oct 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Skgland commented Oct 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kobzol commented Oct 30, 2023

Uh oh!

tgross35 commented Oct 30, 2023

Uh oh!

g2p commented Oct 30, 2023

Uh oh!

tgross35 commented Oct 31, 2023

Uh oh!

lqd commented Oct 31, 2023

Uh oh!

Uh oh!

bjorn3 commented Feb 4, 2021 •

edited

Loading

tgross35 commented Oct 30, 2023 •

edited

Loading

Skgland commented Oct 30, 2023 •

edited

Loading