Cross compilation

• 1 The simple case
• 2 The hard way
  • 2.1 Quirks
  • 2.2 The blue Turris
  • 2.3 Omnia
  • 2.4 The nix dependency

The goal is to run the aggregator on our routers. Therefore, we need to cross-compile it for the target architecture.

1 The simple case

Often, the target architecture will be directly supported by an existing rustc target. This is the case of the Turris Omnia router (with the downside of being statically linked, therefore creating a slightly larger binary). In the case of the Omnia router, the target name is armv7-unknown-linux-musleabihf, for other targets pick one appropriate.

The process assumes the rust compiler to be installed by rustup. We simply ask rustup to install base libraries for the target and let the compiler do the magic.

rustup target add armv7-unknown-linux-musleabihf
cargo build --release --target armv7-unknown-linux-musleabihf

Then, the binary target/armv7-unknown-linux-musleabihf/release/pakon-aggregator is the thing that can run on the router.

2 The hard way

It is possible to compile for whatever linux-based system provided a C cross-compiler is available and LLVM can generate code for the given compiler. It however needs some more work.

We'll need these ingredients:

• The C cross-compiler with whatever needed libraries ready for use.
• Nightly rust compiler. It can be installed with rustup default nightly. It is also possible to use a specific pinned version of nightly (eg. once it is known it works with a given version, it is not needed to risk breaking the process with a newer one).
• Sources for the rust standard library rustup component add rust-src.
• The xargo utility. cargo install xargo. This might also be pinned to a specific version (and it is generally a good idea to take a version from around the same time as the nightly compiler).

Now, we'll do few things.

• Write a target specification. It describes the properties of a target and is a file with JSON in it. While it is mostly undocumented (and the best documentation for the fields is the compiler source code), it can be based on an existing target. To get the example, run rustc -Z unstable-options --print target-spec-json --target armv7-unknown-linux-musleabihf. Place the file into the source code top level directory (or to a path specified by the RUST_TARGET_PATH environment variable), as name-of-the-target.json.
• Define needed environment variables:
  • CC_name_of_the_target to specify the C cross compiler.
  • CFLAGS_name_of_the_target to specify flags for the compiler needed for the cross compilation.
• Edit Xargo.toml in the top-level source code directory and add section for the given target.
• Run xargo build --release --target name-of-the-target. This acts similar as cargo, but first builds the rust standard library for the target instead of using a downloaded one. That's the reason why we needed the source codes and the C cross compiler. It then proceeds to building the application itself, using this standard library.
• If everything went well, the binary is at the usual location (same as with the easy way).

2.1 Quirks

Cross-compilation is tricky business. Therefore, each target usually needs a trick or two.

2.2 The blue Turris

Currently, the old Turris router uses uclibc. We could define a new target for it and do the full hard way process, maybe even needing to provide support for uclibc in some of the core rust libraries. But it seems uclibc and glibc are API-level compatible. Therefore, we can abuse the powerpc-unknown-linux-gnu target and skip writing the target specification. However, we still need to do the xargo build as uclibc is not ABI compatible. Do not download the target libraries through rustup. When we link the rust libraries against uclibc instead of glibc ourselves (through xargo), it seems to work.

We still may want to go the proper way and define a new target specification in the future, just to make sure.

This is what the Makefile in the top-level directory does (with some small workaround).

2.3 Omnia

There's a slight problem. Rust's musl targets are fully statically linked. If the size of the binary is a concern, it is possible to define a custom target that links to musl dynamically. This is a bit tricky, though, as this is not very well supported yet.

There's a working prototype for cross-compiling a dynamically-allocated musl application for Omnia at https://gitlab.labs.nic.cz/mvaner/xcompile. It's only a hello-world application, but it should be applicable to the aggregator as well. The cross-compilation is run by the build script.

2.4 The nix dependency

The nix crate, which is depended on by some of our dependencies (AFAIK something of logging) had a bug which prevented it from working on powerpc. The Cargo.toml currently replaces the dependency with a fixed version. We may drop the replacement once the newest version is used by the other dependencies.

The other option is stripping down and migrating from slog to the basic log crate. This means less fancy log output, but also smaller binary output, so it might be worth it.