Actually Portable Executables

I came across Cosmopolitan on Hacker News, and I was initially confused, due to a few memories of cross-compilation nightmares: while it should be possible to compile for the same architecture regardless of operating system, wouldn’t the OS get confused by the leading bytes of the executable? I read the article explaining how it works, but most of it went over my head.

The example on the Github README used the following script for compilation:

gcc -g -O -static -nostdlib -nostdinc -fno-pie -no-pie -mno-red-zone \
  -o hello.com.dbg hello.c -fuse-ld=bfd -Wl,-T,ape.lds \
  -include cosmopolitan.h crt.o ape.o cosmopolitan.a
objcopy -S -O binary hello.com.dbg hello.com

I converted it into a simple Makefile to run the compilation commands. I tried a bunch of simple C programs (basic arithmetic, reading and writing to files) on Linux+Windows (compiled on Linux), and all of them worked.

Compiling Lua with Cosmopolitan

I decided to try compiling a high-level language built on C. I originally picked Python, but the Makefile for Python seemed too complicated to mess with, so I then picked Lua, which looked much simpler in comparison.

I started out by blindly copy-pasting the flags and includes used in the sample compilation on Github. Ah, it would have been wonderful for my laziness if it compiled out of the box. Following is a play-by-play commentary of trying to compile Lua.

First problem I ran into was header clashes: if I didn’t put -nostdlib -nostdinc while compiling each object file, -include cosmopolitan.h would clash with the system headers. But blocking the system headers meant I would have to change every #include of a system header. I created a bunch of dummy headers with the same names as those in the C stdlib and and included to those instead.

Naming clashes: some of the macros in cosmopolitan.h clashed with macro/function names in Lua: reverse and isempty. I changed the Lua source to avoid this.

A macro FIRST_RESERVED was broken because UCHAR_MAX was missing. I thought UCHAR_MAX was supposed to be in limits.h – the limits.h part of cosmopolitan.h did not have UCHAR_MAX (It had SCHAR_MAX, though.) I added in a #define stating UCHAR_MAX as __UINT8_MAX__ (ie 255).

The default Lua Makefile attempts to use _setjmp/_longjmp in ldo.c when on Linux. I disabled the LUA_USE_LINUX flag for compiling the object files, but this caused an issue with tmpnam in loslib.c (mkstemp is available in Cosmopolitan). I changed the Lua source to use setjmp/longjmp. A similar issue showed in lauxlib.c for sys/wait.h (which is a no-op in non-POSIX systems, as per the Lua source code), and in liolib for sys/types.h so disabled LUA_USE_POSIX over there as well.

The localeconv() function (part of locale.h) was not implemented in cosmopolitan.h, and this caused an error while compiling lobject.c (macro lua_getlocaledecpoint() depended on localeconv()). Changed the macro to just return '.'.

The panic function in Lua static int panic (lua_state*) clashed with that in Cosmopolitan void panic(void). Renamed the lua function to lua_panic. This triggered an error where the panic function was being called in luaL_newstate, so I changed the name there as well.

luaL_loadfilex caused a frame size error – I have never seen this before. A quick internet search shows that this is because a large buffer is allocated on stack when entering the function, and yes, luaL_loadfilex allocates a loadF object containing a char buffer of BUFSIZ. I reduced the size of the buffer to BUFSIZ - 64.

loslib.c requires the setlocale() and LC_* from locale.h, which is defined as an extern value in cosmopolitan.h, but that definition is somehow not enough.. screw it, I just disabled os_setlocale in loslib.c, and then it compiles.

Linking the object files

Ok, time for linking …

gcc -std=gnu99 -o lua   lua.o liblua.a -lm -Wl,-E -ldl
/usr/bin/ld: errno: TLS definition in //lib/x86_64-linux-gnu/libc.so.6 section
.tbss mismatches non-TLS reference in liblua.a(lauxlib.o)
/usr/bin/ld: //lib/x86_64-linux-gnu/libc.so.6: error adding symbols: bad value
collect2: error: ld returned 1 exit status

I forgot, I shouldn’t -lm or -ldl. Ok, let’s try with all the object files instead of liblua.a:

/usr/bin/ld.bfd: lvm.o: in function `l_strcmp':
lvm.c:(.text+0x59): undefined reference to `strcoll'
/usr/bin/ld.bfd: lmathlib.o: in function `math_tanh':
lmathlib.c:(.text+0x21f): undefined reference to `tanh'
/usr/bin/ld.bfd: lmathlib.o: in function `math_sinh':
lmathlib.c:(.text+0x24f): undefined reference to `sinh'
/usr/bin/ld.bfd: lmathlib.o: in function `math_cosh':
lmathlib.c:(.text+0x27f): undefined reference to `cosh'
collect2: error: ld returned 1 exit status

Umm… okay, it looks like some of the functions defined in the cosmopolitan header are yet to be implemented in the static library. That’s okay, I can just quickly fill in the math functions, and I’ll comment out strcoll for now, just because I want to see it compile…. and it successfully compiles!! Let’s run objcopy before trying it out on a system though.

$ objcopy -S -O binary lua lua.exe
$ ls -al
-rwxr-xr-x 1 1953720 Feb 27 01:33 lua
-rwxr-xr-x 1 344064 Feb 27 01:39 lua.exe

That size reduction seems a little too drastic, but let’s see if it runs on Linux:

Awesome. How about Windows?

Summary: it is actually portable

This is pretty incredible: I just had to modify a few lines in a Makefile and some C source files, and I got a Lua executable that works both on Linux and Windows (and possibly others as well). Granted, there are still some details to be filled out (floating point calculation above prints a g), but Cosmopolitan is currently at release 0.2, so there is a lot of time. (Update: Lua is vendored as part of the Cosmopolitan repo since March 8 2021, and is used for dynamic pages.)

Hopefully this means that other languages that have source code completely in C can also be compiled once and run anywhere. Actually Portable Python next, maybe?