bazel: Build or pull in `clang` and `lld` using Bazel

[SlyMarbo]

Currently, we rely on the host having Clang installed at /usr/bin/clang and LLD at /usr/bin/ld, as defined in x86_64_cc_toolchain.bzl. This works, but we should be using the host as little as possible. Furthermore, both tools are often installed in different places, such as /usr/local/bin/clang or /usr/bin/lld. We should either build both tools using the host tooling, or pull in built versions of both.

These may be good starting points:

• grailbio/bazel-toolchain
• Leverage LLVM bazel support bazel-contrib/toolchains_llvm#130

[SlyMarbo]

Also note that rules_rust includes a copy of rust-lld at @rust_linux_x86_64//lib/rustlib/x86_64-unknown-linux-gnu/bin:rust-lld. It's not available directly, but it's included in the @rust_linux_x86_64//:rustc_lib file group, so we could fish it out of there.

[neqochan]

If it is of interest for you, we build lld here by pulling it from the llvm bazel overlay.

[SlyMarbo]

@neqochan Thanks, this is really helpful! I hadn't come across the overlay. I'll see how I get on.

[SlyMarbo]

I've spent the day trying and failing a couple of different approaches. Here are my notes to save some time in the future.

Attempt 1: Download clang+llvm

Start by downloading a recent release of clang+llvm as an http_archive. Sample release listed here. For example:

http_archive(
    name = "clang",
    build_file = "//bazel/third_party:clang.BUILD",
    sha256 = "2c2fb857af97f41a5032e9ecadf7f78d3eff389a5cd3c9ec620d24f134ceb3c8",
    strip_prefix = "clang+llvm-13.0.0-x86_64-linux-gnu-ubuntu-20.04",
    urls = ["https://github.com/llvm/llvm-project/releases/download/llvmorg-13.0.0/clang+llvm-13.0.0-x86_64-linux-gnu-ubuntu-20.04.tar.xz"],
)

To make the binaries externally-usable, we inject the following BUILD file:

exports_files(
    [
        "bin/clang",
        "bin/clang++",
        "bin/llvm-ar",
        "bin/ld.lld",
        "bin/llvm-cov",
        "bin/llvm-nm",
        "bin/llvm-objdump",
        "bin/llvm-strip",
    ],
    visibility = ["//visibility:public"],
)

We also need a filegroup containing the binaries to create a dependency relationship between the cc_toolchain and the clang binaries. This is added to x86_64_cc_toolchain:

redirects_name = "%s_redirects" % name
native.filegroup(
    name = redirects_name,
    srcs = [
        "redirect.sh",
        "@clang//:bin/llvm-ar",
        "@clang//:bin/clang",
        "@clang//:bin/clang++",
        "@clang//:bin/ld.lld",
        "@clang//:bin/llvm-cov",
        "@clang//:bin/llvm-nm",
        "@clang//:bin/llvm-objdump",
        "@clang//:bin/llvm-strip",
    ],
)

The download is quite large (~500 MiB) and takes a while to extract. You then need to make a redirection script in the cross-compiling package (as the binaries need to be in the same package as the cc_toolchain. The script then redirects to the corresponding tool:

#!/bin/bash
set -e
external/clang/bin/ld.lld "$@"

This probably would have worked but I had various issues with shared libraries for ld.lld and clang. If it's causing problems on my laptop, it probably will in others. It's probably simpler to lose some hermeticity and install clang+lld using the local package manager. If anyone knows where statically compiled releases can be found (or to use the bundled shared libraries perhaps?), that may make this the simplest solution.

Attempt 2: Using rust-lld bundled with rules_rust

After struggling with attempt 1 for a while, I remembered that rules_rust bundles a copy of rust-lld. I successfully got the build executing using a similar redirect script:

#!/bin/bash
set -e
external/rust_linux_x86_64/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld "$@"

The downside here is that I couldn't convince the combination of rustc (driven by rules_rust) and rust-lld to build the kernel with static linking. Without that, it won't boot. Getting this working would be a particularly nice solution, as it doesn't mean downloading any extra tools.

[dzbarsky]

https://github.com/dzbarsky/static-clang may be helpful