Add support for statically-linked musl libc Linux targets (-unknown-linux-musl)

[briansmith]

No description provided.

[gnzlbg]

Why is removing the libc crate dependency a goal ?

[cbiffle]

FYI - from updates on rust-lang/libc#659, getrandom is now available in libc on most Linux platforms. The holdout is not musl, but android/Bionic. I suggest decoupling the two parts of this issue an moving the getrandom portion into a separate bug.

[briansmith]

More discussion in #935

[briansmith]

@aig787's PR #1067 added the -musl targets for Linux x86_64 and i686 to CI/CD.

Still to do:

• Enabling static-pie for musl rust-lang/rust#70740 indicates that we need to verify that we're doing the right thing w.r.t. PIE (position independent code), especially when compiling the C and assembly code.
• Add Aarch64 musl targets to CI/CD.
• Minimize the dependencies installed. Currently the i686 job installs "libc6-dev-i386" and "gcc-multilib", but neither of those should be needed for -musl. (No longer relevant.)

[briansmith]

• i686-unknown-linux-musl gives builds sizeof(int64_t) == 4 on 64-bit hosts rust-lang/cc-rs#436 indicates that we should be doing more sanity checks to make sure we're not using the wrong header files. e.g. statically asserting that sizeof(uint64_t) == 8.

rust-lang/rust#70740 indicates that we need to verify that we're doing the right thing w.r.t. PIE (position independent code), especially when compiling the C and assembly code.

See also rust-lang/cc-rs#165. We should have a step in CI/CD that verifies that we support PIE correctly.

[briansmith]

• Minimize the dependencies installed. Currently the i686 job installs "libc6-dev-i386" and "gcc-multilib", but neither of those should be needed for -musl.

I attempted to remove the gcc-multilib dependency but the build fails with:

 running "clang" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "--target=i686-unknown-linux-musl" "-I" "include" "-Wall" "-Wextra" "-std=c1x" "-Wbad-function-cast" "-Wnested-externs" "-Wstrict-prototypes" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wconversion" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wsign-conversion" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-fstack-protector" "-g3" "-Werror" "-U_FORTIFY_SOURCE" "-c" "-o/home/travis/build/briansmith/ring/target/i686-unknown-linux-musl/debug/build/ring-bb6253d99512d637/out/aes_nohw.o" "crypto/fipsmodule/aes/aes_nohw.c"
  In file included from crypto/fipsmodule/aes/aes_nohw.c:15:
  In file included from include/GFp/aes.h:52:
  In file included from include/GFp/base.h:66:
  In file included from /usr/local/clang-7.0.0/lib/clang/7.0.0/include/stdint.h:63:
  /usr/include/stdint.h:26:10: fatal error: 'bits/libc-header-start.h' file not found
  #include <bits/libc-header-start.h>
           ^~~~~~~~~~~~~~~~~~~~~~~~~~

I doubt using gcc-multilib with musl is the right thing to do but I could be convinced otherwise. I'll try experimenting with using --nostdlibinc and/or -nostdlib but it isn't clear that musl libc supports that configuration either.

Help understanding and/or resolving this is appreciated. Otherwise, maybe this is a good motivation to get rid of all the remaining C code in ring.

[briansmith]

I'll try experimenting with using --nostdlibinc and/or -nostdlib

I did the experiment and that seems like it will work fine. I actually had done the same something similar for wasm32-unknown already and I verified that the same works for musl, without gcc-multilib and libc6-dev-i386. I need to clean up the experiment before creating a PR for it.

[briansmith]

PR #1118 makes progress here; see that PR for details.

Remaining work to be done:

• Add static analysis to the build system that verifies that the tests do not link to any shared libraries.
• Run the tests for -musl targets
• Find a solution for x86_64 targets. I'm leaning towards merging PR Pull the integrated assembly x86-64 ChaCha20-Poly1305 implementation from BoringSSL #1111 so that we wouldn't have a real need for poly1305_vec.c, and then removing poly1305_vec.c. Then we can remove the exception for x86_64 targets in build.rs and we should be good to go.

[briansmith]

PR #1133 shows how to use RUSTFLAGS="-Clink-self-contained=yes -Clinker=rust-lld" for linking al -musl targets. For x86_64 (only) we still need a sysroot with header files; for other architectures this should be working fine.

[sylvain101010]

I doubt using gcc-multilib with musl is the right thing to do but I could be convinced otherwise. I'll try experimenting with using --nostdlibinc and/or -nostdlib but it isn't clear that musl libc supports that configuration either.
Help understanding and/or resolving this is appreciated. Otherwise, maybe this is a good motivation to get rid of all the remaining C code in ring.

Hi,
I'm not sure if it will solve this problem, but in case it can help: I have no problem compiling dependencies relying on rustls (and thus ring) for the x86_64-unknown-linux-musl target from a rust:latest Docker image by running apt update && apt install -y musl-tools (https://github.com/skerkour/archive/blob/4a408405978ae714a7be9e92e2105324339f1f7b/simpleserver-rs/Dockerfile#L8)

[briansmith]

I'm not sure if it will solve this problem, but in case it can help: I have no problem compiling dependencies relying on rustls (and thus ring) for the x86_64-unknown-linux-musl target from a rust:latest Docker image by running apt update && apt install -y musl-tools (https://github.com/skerkour/archive/blob/4a408405978ae714a7be9e92e2105324339f1f7b/simpleserver-rs/Dockerfile#L8)

Are you then using the TARGET_CC=musl-gcc TARGET_AR=musl-ar or similar? Are you using RUSTFLAGS="-Clink-self-contained=yes -Clinker=rust-lld" for linking?

[sylvain101010]

No flags are used, the build command is as simple as cargo build --target x86_64-unknown-linux-musl --release

Are those flags special ring requirements, or are they workarounds to have MUSL compile?

[briansmith]

No flags are used, the build command is as simple as cargo build --target x86_64-unknown-linux-musl --release

Are those flags special ring requirements, or are they workarounds to have MUSL compile?

cc-rs will automatically use the musl tools (musl-cc) by default, so if you are happy with using musl-tools, then you will get the correct results.

If you want to use clang and rust-lld and "self contained" linking instead, then you do have to set up some things as described in ring's mk/cargo.sh.

Maybe I should change ring's CI/CD to test the "Just use the defaults" code path.

[ylxdzsw]

Just want to add that if static linking is the only cencern, I'm able to produce a static linked binary using RUSTFLAGS="-C target-feature=+crt-static" cargo build --target x86_64-unknown-linux-gnu --release with glibc. I only use a subset of functions of ring though (namely hmac). Not sure would it affect linking.

[BratSinot]

Hi @briansmith, any news on this issue?

[briansmith]

This has been working for a long time, AFAICT. We have aarch64-unknown-linux-musl, armv7-unknown-linux-musleabihf, i686-unknown-linux-musl, and x86_64-unknown-linux-musl tested in every CI run.