Support `lld` on pure wasm backend

[aturon]

Currently we avoid the need for any linked on wasm32-unknown-unknown, by combining all Rust code into a single compilation unit. This allows us to work around the immaturity of the lld story for wasm, but at the cost of compile times. This issue tracks the status of the lld readiness and rustc's ability to use it.

[alexcrichton]

Awhile back I filed https://bugs.llvm.org/show_bug.cgi?id=35369 with at least one cause of "we can't use lld vanilla upstream". There's a whole lot more issues to solve here, but that's at least one concrete actionable one :)

[pepyakin]

JFYI Experimental LLD support has been landed in Emscripten

WebAssembly/binaryen#1346
emscripten-core/emscripten#6056

[alexcrichton]

I've tested on top of the LLVM 6 upgrade (plus a bit farther on LLVM for a few more patches) with LLD master and everything looks to be working beautifully! Once https://reviews.llvm.org/D42511 lands we won't even need wasm-gc any more, yay!

[alexcrichton]

Ok I've now completed a test run of our entire test suite using LLD to link wasm binaries together. Specifically I was using llvm-mirror/lld@95a9b67 with https://reviews.llvm.org/D42511 applied manually (but that probably wasn't necessary), llvm-mirror/llvm@d14b8f6, and alexcrichton/rust@106657e.

I believe the sequence of steps here are now:

• Wait for LLVM 6 - rustc: Upgrade to LLVM 6 rust-lang/rust#47828
• Add lld as a submodule (turned off by default) in the rust-lang/rust repository.
• Add build system support to build LLD and make its binary available to rustc to run
• Probably upgrade LLVM again (but a much smaller one) necessary to get LLD building
• Land something along the lines of alexcrichton/rust@106657e in rustc to switch wasm's default linker.

I'd hope we can just switch the defaults overnight and not worry about maintaining the old "LTO everything" link mode. There'll proably be bugs but likely nothing that's too hard to backport!

[alexcrichton]

Ok I got a little excited and I've now implemented all of this -- alexcrichton/rust@cbf406f

Once LLVM 6 lands I'll send that as a PR and we should be able to switch to LLD pronto.

[alexcrichton]

PR for this

[alexcrichton]

Landed!

[alexcrichton]

FWIW as a testament to the improvements brought about by LLD related to build times, the wasm32-unknown-unknown builder we have in rust-lang/rust reduced from 1h53m per build to 1h10m per build when this landed, a huge reduction in our build times!

[mgattozzi]

Wow that's a huge reduction. Shaving 40+ minutes off is not a trivial gain at all.