Rust and Cosmopolitan Libc

2023-11-01 Update: I found out that Rust decides system constants like EINVAL and SIGTERM at compile-time, which means that the portability of Rust executables is limited to x86_64-linux and aarch64-linux, the targets with which the executables are built. It’s good to know that Cosmo can be a viable target for Rust, and perhaps later we may use extern system constants in Rust [just like I use them in C][https://github.com/ahgamut/gcc/tree/portcosmo-11.2].

I just built a Rust executable with Cosmopolitan Libc. If you’d like to build it yourself, clone this repo and follow the README – you’ll need a recent gcc, ld.bfd, objcopy, bash, and the latest (nightly) versions of cargo, rustc, and friends.

I’ve been recently getting into Rust, and it seems pretty cool! I’ve also gotten a bunch of software to run on Cosmopolitan Libc over the last year¹, so in June I thought combining Rust with Cosmopolitan Libc would be interesting. Here’s how I got to a hello world! with Rust.

A minimal executable

A good thing about Rust was, unlike Python or Lua, no messing around with C headers. Just find a way to tell cargo to link with cosmopolitan.a at the very end, and you get an APE. I looked up the Rust Embedonomicon and built a no_std example, but it wasn’t that useful – the executable just crashed and I didn’t know if it was on purpose. But the Embedonomicon also described how I could create a custom target for Rust, based on the available targets. Jackpot!

I created a custom target called cosmo.json, based on the existing x86_64-linux-unknown-gnu and x86_64-linux-unknown-musl. It took some trial and error (I set up my own panic handler, but then it clashed with panic_abort, but then it didn’t when I changed the flags to cargo), but eventually I got this simple example below to compile (I took it from some online Rust discussion about the libc crate, I think the code was written by steveklabnik).

#![no_main]
#![no_std]
#![feature(rustc_private)]
extern crate libc;

#[no_mangle]
pub extern "C" fn main(_argc: isize, _argv: *const *const u8) -> isize {
    const HELLO: &'static str = "Hello, world! %d + %d = %d\n\0";
    let x: i32 = 1;
    let y: i32 = 2;
    unsafe {
        libc::printf(HELLO.as_ptr() as *const _, x, y, x+y);
    }
    0
}

#[panic_handler]
fn my_panic(_info: &core::panic::PanicInfo) -> ! {
    loop {}
}

Once I got the libc crate to work, I could do a lot of things. I took a bunch of simple C programs, rewrote them in unsafe Rust with the libc crate, and generally checked if the build process was okay.

Building the std crate

Now I had some simple Rust APEs, but I felt that saying “Rust is Actually Portable” and then showing a bunch of C-like programs with unsafe wouldn’t cut it, so I went on another round of debugging to get the std crate to build.

The std crate pulls in a lot of different crates! There’s core, libc, and alloc, which I used for the above example, panic_abort, panic_unwind, backtrace, and proc_macro, and subcomponents of std itself. I got a tour of the code in the std crate by writing an incomplete target cosmo.json: I’d change a configuration flag, some part of std would break because my flag was wrong, and I’d learn something new about Rust and how std worked.

Once I figured out the right flags in the target cosmo.json, most of the bugs with std went away. The few that remained deserve special mention because it took me a while to fix them.

Weird bugs and their workarounds

One of the weird bugs was internet-related code of std which called the libc crate (like struct hostent or something). I went through Cosmopolitan Libc source code, and everything seemed fine, so the mistake was elsewhere. I modified the std crate trying to figure out what was going on. After a bunch of different compiler errors and searching on the internet, I found the issue was with the filename of my target JSON. So some part of the code or build process uses the filename of the JSON, which was cosmo.json at that time, but it specifically needs a target name like x86_64-linux-unknown-cosmo, otherwise some ifdef cfg_if falls through and breaks a bunch of things. Why does the name of the JSON matter? I don’t know, but I’m happy that this bug doesn’t bother me anymore.

The next “bug” is more a comment about cargo, and is probably because I’m still new to Rust. cargo is a wonderful package manager, and building projects is pretty smooth. But once it comes to building the std crate, some of the convenience disappears, and I’d like a bit more flexibility in specifying what I want cargo to do. I’m building a static executable, and I’d like to say, “okay cargo forget about linking -lunwind or -lm or whatever, just listen to me, cosmopolitan.a has everything you need for this”, but I couldn’t find any combination of flags to communicate this. Eventually I gave up and wrote a bash script which just filtered out all the linker arguments I didn’t want before calling gcc.

The last set of bugs were at the link stage – I had every crate compiling without error, but when linking the executable, I found that I was missing a few symbols. Some were easy to add, like stat64 and __xpg_strerror_r, but the std crate required the pthreads key API (pthread_key_create etc.) to be implemented, which was weird because I thought I had specified single-threaded in cosmo.json. Another dependency of the std crate was backtrace, which requires libunwind in the default linux builds even though there is a noop crate that can be used for std. I couldn’t figure out the right flags to avoid these linker errors, so I wrote some dumb stubs to just get by the linker. A couple of days later, Justine Tunney implemented the pthreads key API in Cosmopolitan Libc, and I asked Justine to add the libunwind stubs as well so backtrace wouldn’t complain. I think there should be a cargo flag or something to choose using the noop crate in backtrace when compiling std.

Anyway, once I got through these linker errors, I downloaded the latest cosmopolitan.zip amalgamation from here, and now I could build some Actually Portable Executables with Rust.

Hello World! and a few examples

Rust is Actually Portable: here’s the hello world program that uses Rust and Cosmopolitan Libc:

fn main() {
    println!("Hello World! This is an APE built with Rust.");
}

That’s it. No unsafe, no changes to the Rust std source code. You just need to provide the right flags to cargo, and done. I also picked the first few examples from Rust By Example to build and try, and they all worked as expected. I’ve not tried a lot of things, so there’s room for experimentation and submitting PRs with Rust code (like with backtrace) and to Cosmopolitan Libc (implementing the libunwind stubs or filling out the pthreads API).

Closing Notes

I feel the package manager and documentation of Rust are big reasons why it is so popular. Being new to Rust, I was able to learn about safe vs unsafe, the libc crate, cargo, the std crate, backtrace, panic, rustc, and what Rust-generated assembly looked like, all over maybe a couple of weekend afternoons. Cosmopolitan Libc provided a unique angle to tour Rust, and it was a lot of fun to discover all of this.

Last March, Lua was the first language to be ported to Cosmopolitan Libc. At that time, parts of the libc API were still missing; when porting Python I submitted PRs for getnameinfo, struct servent etc. so Python’s socket library would work. But since that time a lot of work has been put into Cosmopolitan Libc, so that software built on it can be fun to develop and fast. The almost-complete libc API has made it so much easier to port software – Rust just needed a JSON file and some flags to cargo! I’m pretty sure a lot of software can be built with Cosmopolitan Libc, it’s just a matter of convincing the build system. If someone figures out a way to even partially automate this (say like FreeBSD’s ports), that would be amazing.