I recently ported Leaf from Linux to OSX. I figured it was about time to start making it cross-platform. The goal is to get it compiling useful libraries for iOS and Android this year. Here I describe the steps I took.
Building 3rd party libraries
Thanks to brew it wasn’t much effort getting most of the packages I needed. Though I worried that the gcc package may be broken; what looked like a hang was actually compilation in the background — roughly an hour without any kind of feedback. I watched in top to at least make sure it was doing something.
I had issues with Python versions. I took a step back and migrated some Python 2 code to 3 in order to resolve them. I also had to add PYTHONIOENCODING="utf-8" to the environment, which seems wrong since my environment already has appropriate variables for encoding. It is the Leaf tests that depend on Python. I’m using my shelljob module as a job manager to run tests in parallel.
scons, which is used for builds, is also Python based, but it seemed to run without issue.
The biggest dependency is LLVM. It’s an essential part that produces the final machine code. Fortunately that just worked, despite taking a long time to compile on my MacBook Pro (it’s not really a good development machine).
Fixing the build system
I’m a bit disappointed in scons support for cross-platform builds. Either it’s not documented well, or the support isn’t very good.
The only reliable thing I found is PLATFORM, set to darwin on OSX and posix on Linux. I use them to do several if statements in the build files. I suppose I’ll have to revisit those as I add more supported platforms.
1 2 3 4 5 6 |
baseEnv["IS_DARWIN"] = baseEnv['PLATFORM'] == "darwin" if baseEnv["IS_DARWIN"]: baseEnv.Append(CPPDEFINES = '-DIS_DARWIN') baseEnv["IS_POSIX"] = baseEnv['PLATFORM'] == "posix" if baseEnv["IS_POSIX"]: baseEnv.Append(CPPDEFINES = '-DIS_POSIX') |
Warnings
I’m using GCC on both platforms, but the versions differ. I had to modify the warning flags to clean up the output. I added -Wno-sign-conversion since the warning would otherwise generate a lot of warnings. Mainly these relate to using unsigned for enum values, but standard operations end up making them int instead. It’s an ugly relic of C’s history.
I added -Wno-unused-private-field, but that seems like a good warning. I’ll have to clean up the warnings and allow the message again later. I also used the new -isystem for some include directories to clean up warnings coming from library code.
One more I added was -Wno-potentially-evaluated-expression. I was getting a warning that didn’t make sense, and a quick search indicated this may be a bug in GCC.
Libraries
I had to add a variable GMP_DIR and manually locate the libgmp install. It seems a bit odd that scons can’t find the stuff installed with brew. The problem with adding a GMP_DIR is now scons doesn’t attempt to look for the location at all on its own, and a default must be specified, even on Linux where it could find it before. This seems a bit wrong but I’m not sure how to fix that.
1 |
PathVariable('GMP_DIR','Where GMP is installed', '/usr/' ), |
I was a bit surprised that the only modification to the actual list of libraries I needed was removing rt and dl from the list. It seems to work on OSX without them, so they must be part of some standard set.
I needed to change the structure of my libraries. I had two libraries that depended on each other. This is fine on Linux, but OSX seems to require a strict one-way relationship. It also requires that the symbols in the library are fully known at library link time. Fortunately I didn’t have to change much here. There is only one two-way dependency. It’s also not a nice design and I should remove it (part of a shared data structure used by parsing and typing).
Initial code Changes
Now comes the annoying part. This first thing I’m hit with are errors about struct and class mismatches. This happens when you declare something class and then forward declare it as struct, or vice-versa. The standard says this shouldn’t make a difference, yet some compilers seem to dislike it. I recall MSVC would actually link incorrectly if you mismatched these. It’s very annoying though, I shouldn’t have to know whether something was declared struct or class in order to forward declare it.
I added #include <vector> to many files. This is one of those oddities of standard headers. On some platforms they include other standard headers, and some they don’t.
That was enough to get it compiling and have most of the unit tests work. One that failed was actually a C++ feature test. I added it since I relied on then new features and I wanted to know immediately if the compiler wasn’t supporting it correctly. And it failed!
But it was my fault. I was lacking a constructor to initialize fields. This is one aspect of many languages that annoy me, the ability to have undefined values by default. Worse, since many platforms end up with zero defaults you just don’t notice these errors immediately.
Broken env utility
The top of my Python files had the standard #! line:
1 |
#!/usr/bin/env python3
|
Somebody once told me this was the correct way to do this, rather than linking directly to python3. The above will properly lookup the executable in the path.
The problem is that Apple broke this utility. Despite it’s documentation clearly stating it forwards the environment, the version on OSX strips out certain values: things like LD_LIBRARY_PATH. While working on a project that involves compiling libraries, it’s a critical variable.
The workaround here is kind of ugly. Leaf has a env.sh file that is sourced to setup the environment. It now additionally adds variables like APPLE_LD_LIBRARY_PATH. The Python scripts then search for such variables and patch the environment.
Linking changes for Leaf programs
I fully expected that linking programs on OSX would not be the same as on Linux. Leaf is already using a gcc wrapper, which makes it easier. It was rather painless to figure out a set of options that get it working. It’s likely this needs to be revisited anyway once Leaf starts producing production code.
One surprise was the lack of a -fini option to the linker. This is how you specify the code that should be run when a shared library is unloaded. OSX just doesn’t support this. There appears to be no way to run code when unloading a library!
1 2 3 4 5 6 7 8 |
#if defined(IS_DARWIN) std::string cmd( "gcc -dynamiclib -Wl,-undefined,dynamic_lookup,-o," ); cmd += mloader->dyn_name(path_base) + ",-init," + "_" + *module_name + platform::export_sep + "_init_module" //there just is no "fini" on OSX! //+ " " + mloader->resolve_dyn_core( "runtime" ) + " " + mloader->obj_name(path_base); #else |
That’s it!
The port to OSX took less effort than I expected.
One big change to Leaf last year that made it easier was the dropping of zero cost exceptions. It was a major change, and I felt kind of bad since it wasn’t easy to get it working. But I knew it wasn’t cross-platform compatible, and I have my suspicions it wouldn’t be efficient for Leaf error handling. Not using a “native” exception model means there is no effort involved in porting it to new platforms. A huge bonus.
It was also made easier by having the same chip architecture. I didn’t have to change any variable types in the output. It would probably have been more work to get a 32-bit program compiled on linux than porting to OSX. This is probably my next step. It’ll be the base work of having a cross-target compiler.
