Build Systems

When it comes to compiling code and building software applications and projects, you can run the commands yourself, or you can script up something custom, or use an existing build system. Build systems tend to not be specific to any operating system, but can be specific to a programming language, and if you are seeking to write some code and build it, then you should consider using one of the following build systems, of course, depending on your coding language of choice, as build systems due vary based on the language, more than they vary based on the computer architecture or operating system.

When you compile code, it is specific to both the computer architecture and operating system, depending on the coding language as some like Java, Python, and others are architecture and/or operating system independent, while others such as C, C++, and other compiled to binary languages, are computer architecture and operating system specific. As such, they require changes to be build on different computer architectures and operating systems, and most anything being built has dependencies that requires building using or to linking to other software, and then, once it is all built, must be installed for usage.

To reduce the burden and make all this feasible and manageable, various build systems have been developed, and continue to be developed. While some can adapt to many different scenarios and combinations, even many languages, there is a time and place for each, or more accurately, a recommended build system language for each language. This by no means is comprehensive, covering all build system, but will focus on the main ones used in FOSS Linux applications and software. Most all the software you will run on Linux or Unix was built using one of the following, if not all and more.

The following are all Open-Source build systems and software tools, that can be used for building both Open- and Closed-Source projects, respective of licenses. In your FOSS and Linux career, you are likely to come across at least one if not all, and you are likely using software build by them as well.

Front-End

Technically, all of the following could be considered a back-end, but for our purposes, we are going to consider the following as build system front-ends, but really they are meta-build tools. This is due to the fact that this is the interface you will be using, the files you will create and work with as part of the instructions and steps to building your code and project. Each of them will in turn, use a back-end to create the actual files that run the commands to compile the code and build the project, including installation and removal of the resulting build artifacts. While there are many more, some language specific, we are going to focus on the following.

These are the primary build systems used in the vast majority of open-source software, and as such, you should be familiar with them at minimum. Wikipedia lists many more and maintains a List of build automation software , with the ones being covered here under the Meta build section. No this has nothing to with Meta the company, whose build system is called Buck .

None of these actually build or compile the code into software, they just generate files that are run separately that do just that, these prepare and generate those tedious, and at times, rather large files full of commands being run to build the software and compile the code.

Meson

meson logo

Meson is one of the newest build systems first appearing in 2013 under the BSD-3-Clause license, written in Python, and it has taken the open-source world by storm, with numerous projects, especially, projects that are written in C, have migrated to Meson; pretty much all of the Gnome Foundation's software. While Meson supports and works with a variety of languages, it is best for building software written in C. Therefore, it is only recommended to use Meson when compiling C code, but others may have a difference of opinion, and Meson supports several languages C, C++, C#, CUDA, Objective-C, D, Fortran, Java, Rust, and Vala.

Meson, being written in Python, allows for usage of Python syntax and code, which makes it very flexible and powerful, and for people familiar with Python, heavily reduces the learning curve. Despite being written in Python, Meson performs at least as well as others, and in some cases, outperforms them, and in many cases Meson is faster than building with things like GNU Autotools. But, the build system really has no impact on the actual time to compile, that is system resource dependent, it is the time expended in configuring and preparing the build system.

For our purposes, Meson only supports the Ninja back-end, being a modern build system, Meson has no support for Make or Makefiles, it will not generate them, and by default, it generates Ninja build files. Meson is not intended for cross-architecture or cross-platform building, but it can be used as a backend to build Python wheels, via the meson-python package. Meson should be your starting point anywhere Meson is suitable, like software written in C.

CMake

cmake logo

CMake is the most popular modern cross-platform build systems first appearing in 2000 under the BSD-3-Clause license, written in C and C++, and is one of the most widely used build systems on both Linux and Windows; used by most all KDE and QT software. While CMake supports and works with a variety of languages, it is best for building software written in C or C++. Therefore, it is only recommended to use CMake when compiling C and/or C++ code, but others may have a difference of opinion, and CMake supports many programming languages.

CMake comes with a bit of a learning curve, with basic usage being straight forward, but it has its own language syntax.

For our purposes, Cmake only supports Ninja and Make back-ends, being a modern intermediary build system, CMake has support for Make or Makefiles, that it generates by default, and optionally can generate Ninja build files via a parameter. 

$ cmake -G Ninja ./

Otherwise, when a parameter is not specified, CMake generates a Makefile, by default. This can be used to see the actual time difference for a given codebase when compiled via Make vs Ninja, and is likely to be negligible.

GNU Autotools

cmake logo

GNU Autotools is one of the oldest build systems dating back to 1990s and it's suite of tools has been used more than just about any other software and technology that is used for building software. GNU Autotools is king of cross-architecture and cross-platform building, but given its history, it is arguably, the oldest and most complex and cumbersome build system with a steep learning curve, but, there is little that GNU Autotools cannot build; through expending maximum effort.

As it's name implies, GNU Autotools is not just a single piece of software, but a suite of software that works together using more of the Unix Philosophy of simplicity and modularity, programs doing one thing well and working together with others seamlessly. GNU Autools consists of three primary components.

Unless a project has a need to support a vast variety of architectures and platforms, most have migrated away from GNU Autotools, but that is not to discount it's importance and utility. GNU Autotools remains in both development and usage, and will continue for the foreseeable future, and it should not be discounted or overlooked. GNU Autotools shines when all others fall short, but it is non-trivial, with a steep, legacy learning curve.

GNU Autotools is not recommended for new projects, again, unless the requirements are cross-architecture and cross-platform, and alternatives are insufficient. GNU Autotools is mostly for the many projects still using that build system, some of which may never migrate away. One of the best resources for Autotools, is the Autotools Mythbuster website resource from Diego Elio “Flameeyes” Pettenò, but given its age, there are countless resources available.

It is possible, that you will find yourself in a situation of either maintaining and/or extending a codebase using GNU Autotools, or potentially, tasked with migrating it to a newer build system such as Cmake or Meson. Therefore, it is good to know, be familiar, and maybe even proficient with GNU Autotools.

Back-End

Technically, the following are just commands, but for our purposes, we are going to consider the following as build system back-ends, since they are the result of the front-end, the meta builders, that generated the files used by the back-end, aka commands make and ninja.

These are the primary build systems used in the vast majority of open-source software, and as such, you should be familiar with them at minimum. Wikipedia lists many more and maintains a List of build automation software , with the ones being covered here under the Sequencing section.

These are the work horses that actually build and compile the code into software, they generate the final software binaries, which they can also install to and/or remove from the system. These primarily operate using generated tedious files, which you can make yourself, and/or run directly against the command, but should be generated by a front-end, meta, build system.

Ninja

Ninja is one of the newest build systems first appearing in 2012 under the Apache License 2.0, written in C++ and Python, with intention of being a replacement for Make. In replacing Make, Ninja has two primary goals, to have the input files generated by a higher-higher level build system, a front-end or meta build system, and to run builds as fast as possible; Ninja is not intended for human generation of it's build files.

Ninja is fast, and it has a cooler name than Make, but that the real time in building is the time spent in compiling, it's performance over Make can be negligible at times, but it can be faster than Make for large projects, where it really shines; adding up the soon to be MIA pennies.

When you have a choice, it is recommend to use Ninja as the back-end from a front-end such as CMake, and if you run into problems, then switch back to Make. You may experiment with the time difference, and given that you are not writing either, the choice in usage of Ninja may not be as important as the front-end choice. Run it when you can, but don't live or die by the sword.

Make

Make is one of the oldest build systems, or software commands, first appearing in 1976 and eventually ended up under GPL, currently GPLv3, written in C. Make has likely built more software than any other build system or tool, and predates Linux, going back to the origins of Unix. Make is not limited to building and can be used to perform just about any operation via the shell, making it extremely flexible and powerful.

Make has a steep learning curve, it is mostly legacy technology, and it is difficult to work with, if you find yourself manually creating or editing the Makefiles, which you can do and is not discouraged as with Ninja. However, given tools like Automake, you are best to use a front-end, rather than creating Makefiles by hand, but you have that choice, so you do you.

Tools

There are a couple other tools that are good to be aware of, as they are used in all front-ends, Meson, CMake, and GNU Autotools, tools resulting from the GNU Autotools suite of software. They are optional and not required. They aide in using and working with shared libraries during build and with internationalization and localization (i18n and l10n) in development and build.

These you should be aware of and use as you can with any of the mentioned build systems, especially, gettext, if you are seeking to support more than a single language and locale.

pkg-config

pkg-config is helper tool for shared libraries first appearing in 2000 under the GPL, written in C. It aids in dependency information, providing paths, flags, and other information for a given shared library. This relies upon shared libraries providing a <library name>.pc file, where the library name may or may not include any lib prefix, as in libMyLibrary or MyLibrary. The tool is basically just providing the fields from the *.pc file for a given library, and there is one file per shared library, and the library name matches the name of the *.pc file.

At times, pkg-config is necessary with Meson and CMake when their built in methods fail, or in CMake's case, a <library name>.cmake file does not exist or cannot be found, you can typically revert back to system pkg-config. Just like pkg-config's *.pc file, CMake relies upon the very same, a *.cmake file. If neither are available, you can make one, or hard code the information in your front-end build system, which is not recommended.

pkg-config, the command, can also be used directly. 

$ gcc -o test test.c $(pkg-config --libs --cflags libpng)

Here pkg-config is providing the dependent libraries and C flags that are required for building and linking against libpng, which you can type yourself if you prefer, or just let pkg-config save you the headache.

gettext

gettext is system for internationalization and localization (i18n and l10n) first appearing in 1990 under various FOSS licenses. The gettext system consists of the command gettext and a share library libintl, that programs build against,with the build system invoking the gettext command. This unlike others, requires inclusion in the source code itself.

Within the source code, various strings of text are wrapped in a gettext("Text in some language") function for i18n and l10n. The default text is usually in English, but this is not a requirement, and you create various files that relate using an identifier, in this case "Text in some language", with translations to other languages and locales. This allows for abstraction of the string text from the codebase for i18n and l10n.

This is only relevant if you intend for your code to serve an international audience, as if it is for local or internal usage, this may not be as applicable or used at all. Not all applications support internationalization and localization, but by default, FOSS ones should.