Skip to content

Conversation

matthiaskrgr
Copy link
Member

Successful merges:

Failed merges:

r? @ghost
@rustbot modify labels: rollup

Create a similar rollup

abrown and others added 17 commits February 11, 2022 17:21
This change adds a flag for configuring control-flow protection in the
LLVM backend. In Clang, this flag is exposed as `-fcf-protection` with
options `none|branch|return|full`. This convention is followed for
`rustc`, though as a codegen option: `rustc -Z
cf-protection=<none|branch|return|full>`.

Co-authored-by: BlackHoleFox <[email protected]>
… r=oli-obk

More informative error message for E0015

Helps with rust-lang#92380
Add support for control-flow protection

This change adds a flag for configuring control-flow protection in the LLVM backend. In Clang, this flag is exposed as `-fcf-protection` with options `none|branch|return|full`. This convention is followed for `rustc`, though as a codegen option: `rustc -C cf-protection=<none|branch|return|full>`.

There are a couple of issues with this PR as-is, but I figured it would be advantageous to get some review feedback before proceeding much further:
 - [x] I would like this change would have some kind of test, e.g., compile a simple example with cf-protection enabled and verify that the ELF output indeed has the `IBT` and `SHSTK` issues; I will need someone to point out where such a test would go as I'm not too familiar with the `rustc` repository
 - [ ] Any assembly code in the libraries will need to be manually checked to see to it that when this flag is set, `ENDBR*` instructions are inserted in the right places; again, I will need a bit of help with this, since I don't know where all the x86 assembly lives [edit: see followup issue]
 - [x] A more serious issue, I think: if other libraries are not also built with these flags enabled, then LLVM will seek the "lowest-common denominator" and compile without cf-protection enabled; what I observed is that compiling a simple example with `-C cf-protection=full` doesn't result in the correct ELF tags, presumably because the standard libraries were not compiled with the same flags--any thoughts on this?
…rk-Simulacrum

bootstrap.py: Suggest disabling download-ci-llvm option if url fails to download

I got an error when trying to build the compiler using an old commit, and it turns out it was because the option `download-ci-llvm` was implicitly set to true. So this pull request tries to add a help message for other people that may run into the same problem.

To reproduce my error:

```
git checkout 8d7707f
./x.py test
[...]
spurious failure, trying again
downloading https://ci-artifacts.rust-lang.org/rustc-builds/db002a06ae9154a35d410550bc5132df883d7baa/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
curl: (22) The requested URL returned error: 404

failed to run: curl -# -y 30 -Y 10 --connect-timeout 30 --retry 3 -Sf -o /tmp/tmp8g13rb4n https://ci-artifacts.rust-lang.org/rustc-builds/db002a06ae9154a35d410550bc5132df883d7baa/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
Build completed unsuccessfully in 0:00:46
```

This is my `config.toml`:

```
# Includes one of the default files in src/bootstrap/defaults
profile = "compiler"
changelog-seen = 2

[rust]
debug = true
```

To reproduce an error with this branch:

Change line 618 of bootstrap.py to
```
        url = "rustc-builds-error404/{}".format(llvm_sha)
```

Delete llvm and cached tarball, and set `llvm.download-ci-llvm=true` in config.toml.

```
./x.py test
[...]
spurious failure, trying again
downloading https://ci-artifacts.rust-lang.org/rustc-builds-error404/719b04ca99be0c78e09a8ec5e2eda082a5d8ccae/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
curl: (22) The requested URL returned error: 404

failed to run: curl -# -y 30 -Y 10 --connect-timeout 30 --retry 3 -Sf -o /tmp/tmpesl1ydvo https://ci-artifacts.rust-lang.org/rustc-builds-error404/719b04ca99be0c78e09a8ec5e2eda082a5d8ccae/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
error: failed to download llvm from ci
help: old builds get deleted after a certain time
help: if trying to compile an old commit of rustc, disable `download-ci-llvm` in config.toml:

[llvm]
download-ci-llvm = false

Build completed unsuccessfully in 0:00:01
```

Regarding the implementation, I expected to be able to use a try/catch block in `_download_ci_llvm`, but the `run` function calls `sys.exit` instead of raising an exception so that's not possible. Also, suggestions for better wording of the help message are welcome.
add link to format_args! when mention it in docs

close rust-lang#93904
update miri

to fix the libcore test suite
r? `@ghost`
@rustbot rustbot added T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. rollup A PR which is a rollup labels Feb 12, 2022
@matthiaskrgr
Copy link
Member Author

@bors r+ rollup=never p=5

@bors
Copy link
Collaborator

bors commented Feb 12, 2022

📌 Commit 555a6fe has been approved by matthiaskrgr

@bors bors added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Feb 12, 2022
@bors
Copy link
Collaborator

bors commented Feb 12, 2022

⌛ Testing commit 555a6fe with merge 6da423dba6f359b4552fec9a6697ee49e50dea1d...

@rust-log-analyzer
Copy link
Collaborator

The job aarch64-gnu failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
test [codegen] codegen/box-maybe-uninit.rs ... ok
test [codegen] codegen/debug-column-msvc.rs ... ignored
test [codegen] codegen/call-llvm-intrinsics.rs ... ok
test [codegen] codegen/c-variadic-opt.rs ... ok
test [codegen] codegen/cf-protection.rs#branch ... FAILED
test [codegen] codegen/cf-protection.rs#none ... FAILED
test [codegen] codegen/cf-protection.rs#full ... FAILED
test [codegen] codegen/cfguard-non-msvc.rs ... ok
test [codegen] codegen/dllimports/main.rs ... ignored
test [codegen] codegen/dllimports/main.rs ... ignored
test [codegen] codegen/cf-protection.rs#undefined ... FAILED
test [codegen] codegen/coercions.rs ... ok
test [codegen] codegen/cf-protection.rs#return ... FAILED
test [codegen] codegen/debug-linkage-name.rs ... ok
test [codegen] codegen/debug-compile-unit-path.rs ... ok
test [codegen] codegen/consts.rs ... ok
test [codegen] codegen/catch-unwind.rs ... ok
---
Some tests failed in compiletest suite=codegen mode=codegen host=aarch64-unknown-linux-gnu target=aarch64-unknown-linux-gnu

failures:

---- [codegen] codegen/cf-protection.rs#branch stdout ----

error in revision `branch`: compilation failed!
status: exit status: 1
command: "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/codegen/cf-protection.rs" "-Zthreads=1" "--cfg" "branch" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen/cf-protection.branch" "-Crpath" "-O" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/aarch64-unknown-linux-gnu/native/rust-test-helpers" "-Z" "cf-protection=branch" "--target" "x86_64-unknown-linux-gnu" "-L" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen/cf-protection.branch/auxiliary" "--emit=llvm-ir"
------------------------------------------

------------------------------------------
stderr:
stderr:
------------------------------------------
error[E0463]: can't find crate for `std`
  |
  = note: the `x86_64-unknown-linux-gnu` target may not be installed
  = help: consider downloading the target with `rustup target add x86_64-unknown-linux-gnu`
  = help: consider building the standard library from source with `cargo build -Zbuild-std`
error: requires `sized` lang_item

error: aborting due to 2 previous errors


For more information about this error, try `rustc --explain E0463`.

------------------------------------------


---- [codegen] codegen/cf-protection.rs#none stdout ----

error in revision `none`: compilation failed!
status: exit status: 1
command: "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/codegen/cf-protection.rs" "-Zthreads=1" "--cfg" "none" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen/cf-protection.none" "-Crpath" "-O" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/aarch64-unknown-linux-gnu/native/rust-test-helpers" "-Z" "cf-protection=none" "--target" "x86_64-unknown-linux-gnu" "-L" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen/cf-protection.none/auxiliary" "--emit=llvm-ir"
------------------------------------------

------------------------------------------
stderr:
stderr:
------------------------------------------
error[E0463]: can't find crate for `std`
  |
  = note: the `x86_64-unknown-linux-gnu` target may not be installed
  = help: consider downloading the target with `rustup target add x86_64-unknown-linux-gnu`
  = help: consider building the standard library from source with `cargo build -Zbuild-std`
error: requires `sized` lang_item

error: aborting due to 2 previous errors


For more information about this error, try `rustc --explain E0463`.

------------------------------------------


---- [codegen] codegen/cf-protection.rs#full stdout ----

error in revision `full`: compilation failed!
status: exit status: 1
command: "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/codegen/cf-protection.rs" "-Zthreads=1" "--cfg" "full" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen/cf-protection.full" "-Crpath" "-O" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/aarch64-unknown-linux-gnu/native/rust-test-helpers" "-Z" "cf-protection=full" "--target" "x86_64-unknown-linux-gnu" "-L" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen/cf-protection.full/auxiliary" "--emit=llvm-ir"
------------------------------------------

------------------------------------------
stderr:
stderr:
------------------------------------------
error[E0463]: can't find crate for `std`
  |
  = note: the `x86_64-unknown-linux-gnu` target may not be installed
  = help: consider downloading the target with `rustup target add x86_64-unknown-linux-gnu`
  = help: consider building the standard library from source with `cargo build -Zbuild-std`
error: requires `sized` lang_item

error: aborting due to 2 previous errors


For more information about this error, try `rustc --explain E0463`.

------------------------------------------


---- [codegen] codegen/cf-protection.rs#undefined stdout ----

error in revision `undefined`: compilation failed!
status: exit status: 1
command: "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/codegen/cf-protection.rs" "-Zthreads=1" "--cfg" "undefined" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen/cf-protection.undefined" "-Crpath" "-O" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/aarch64-unknown-linux-gnu/native/rust-test-helpers" "--target" "x86_64-unknown-linux-gnu" "-L" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen/cf-protection.undefined/auxiliary" "--emit=llvm-ir"
------------------------------------------

------------------------------------------
stderr:
stderr:
------------------------------------------
error[E0463]: can't find crate for `std`
  |
  = note: the `x86_64-unknown-linux-gnu` target may not be installed
  = help: consider downloading the target with `rustup target add x86_64-unknown-linux-gnu`
  = help: consider building the standard library from source with `cargo build -Zbuild-std`
error: requires `sized` lang_item

error: aborting due to 2 previous errors


For more information about this error, try `rustc --explain E0463`.

------------------------------------------


---- [codegen] codegen/cf-protection.rs#return stdout ----

error in revision `return`: compilation failed!
status: exit status: 1
command: "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/codegen/cf-protection.rs" "-Zthreads=1" "--cfg" "return" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen/cf-protection.return" "-Crpath" "-O" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/aarch64-unknown-linux-gnu/native/rust-test-helpers" "-Z" "cf-protection=return" "--target" "x86_64-unknown-linux-gnu" "-L" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen/cf-protection.return/auxiliary" "--emit=llvm-ir"
------------------------------------------

------------------------------------------
stderr:
stderr:
------------------------------------------
error[E0463]: can't find crate for `std`
  |
  = note: the `x86_64-unknown-linux-gnu` target may not be installed
  = help: consider downloading the target with `rustup target add x86_64-unknown-linux-gnu`
  = help: consider building the standard library from source with `cargo build -Zbuild-std`
error: requires `sized` lang_item

error: aborting due to 2 previous errors


For more information about this error, try `rustc --explain E0463`.

------------------------------------------



failures:
    [codegen] codegen/cf-protection.rs#branch
    [codegen] codegen/cf-protection.rs#full
    [codegen] codegen/cf-protection.rs#none
    [codegen] codegen/cf-protection.rs#return
    [codegen] codegen/cf-protection.rs#undefined
test result: FAILED. 256 passed; 5 failed; 65 ignored; 0 measured; 0 filtered out; finished in 4.01s

Build completed unsuccessfully in 0:20:12

@bors
Copy link
Collaborator

bors commented Feb 12, 2022

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Feb 12, 2022
@matthiaskrgr matthiaskrgr deleted the rollup-wdajqnc branch February 13, 2022 00:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
rollup A PR which is a rollup S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants