Not all languages are made with the same purpose in mind. When it comes to designing a new language like Leaf, it’s important to choose what type of language it will be. While Leaf should function as an effective general purpose language, I’d also like it to fulfill the needs of systems programmers. In my earlier discussions about values this goal plays a very important role.
What is a system language anyway? Clearly I am thinking about something like C, or C++. There are some critical aspects which set these languages apart from say Python or JavaScript. There are also key features which differentiate them from Java and C#. All such fairly general purpose languages are likely to be a valid choice for one project or another. However most languages are not suitable for what I call systems programming. This is the realm of operating systems, device drivers, core libraries, virtual machines, and embedded programming. In this article I’d like to go over some of the key features that a system language must support.
Features
- Native Types: The fundamental types of the hardware should be mapped to language types. Though conversions are possible at times, efficient code will need to work directly with native types.
- Platform ABI: Strongly related to native types is a platform ABI. You must be able to call and write both system functions and shared library functions using the platform binary interfaces.
- Inline Assembly: A generic language cannot possibly expose all aspects of the underlying hardware. There will always be cases where you need to call a few specific instructions. For this reason good integration of assembly is required.
- Strict Flow: As much as I believe exceptions are important there will be code where you definitely don’t want them. Strict imperative flow with error code checking can actually be the better, cleaner option when working near the bare metal.
- Library Free: It must be possible to write working code without using the standard library. While a few intrinsics will be necessary, the bulk of libraries should be optional.
- No Virtual Machine: The compiled language must run directly against the OS and/or the hardware. It should not require an additional hosting process. This is most essential for the lowest levels of code such as operating systems, drivers, and of course the virtual machines of other languages.
- OS Access: A user-level application must be allowed to access all of the functionality of the kernel. This implies having a type system which is capable of efficiently using kernel exposed structures and functions.
- Signal Handling: If the OS will be reporting signals there has to be a way to handle them. The same applies to structured exception handling or any other alert mechanism. The key here is to provide a well defined environment during the handling.
- Callback Functions: A variety of libraries, and possibly the OS, will be expecting callback functions.
- Memory Management: In many situations a generic allocation scheme will not produce the most efficient code. To achieve the highest performance possible there has to be a way to self-manage object allocation and destruction. This is also necessary in situations without an OS or standard library.
- Memory/Instruction Ordering: For concurrency it is important to be able to define the ordering of certain operations and to have strong guarantees.
- Mutable Value Semantics: It must be possible to modify memory in-place. This is required to make use of shared memory, OS allocated memory blocks, memory mapped files, and for highly efficient data manipulation.
- Bytes: Many of the other requirements imply this one. You will need to work with the raw underlying bytes of types and data structures.
Goal
The goal of a system language isn’t just to make such features available. If that were all then C would suffice — many people still believe it does. I believe a good system language must incorporate these key requirements into a larger, richer, system. C++ is one such language which retains the low-level functionality required of a system language yet provides a lot of high-level constructs to increase productivity.
In a sense, the term “system language” is really short for “general purpose language which allows systems programming”. Though the low-level aspects must be there the language should be also be a productive way to write high level code. You should be able to write clean user interface libraries and build rich applications. It should also be a reasonable platform for developing web servers, machine learning software, networking, games, and for certain kinds of data processing. And certainly it should be the basis for other domain specific languages, to write both the virtual machine and standard libraries.
