Clang

Prerequisites

One of the goals of the Clang project is to be compatible with GCC. Occasionally some packages will fail to build correctly with it and some may build successfully but segfault when executed. Some packages also have GCC specific code and will also fail during compiling. In these events, GCC will need to be used as a fallback.

USE flags

Some packages are aware of the clang USE flag.

<syntaxhighlight lang="bash">USE="clang"</syntaxhighlight>

Emerge

GCC fallback environments

Create a configuration file with a set of environment variables using Portage's built in /etc/portage/env directory. This will override any defaults for any packages that fail to compile with clang. The name used below is just an example, so feel free to choose whatever name is desired for the fallback environment. Be sure to substitute chosen name with the examples used in this article.

<syntaxhighlight lang="bash">
CC="gcc"
CXX="g++"
</syntaxhighlight>

The above is the most basic environmental variable needed. You can change it to suit your needs, such as enabling/disabling link-time optimizations, alternative AR, NM, RANLIB, and so on. Here are two examples below:

<syntaxhighlight lang="bash">
CC="gcc"
CXX="g++"
CFLAGS="-flto -march=native -O2 -pipe"
CXXFLAGS="${CFLAGS}"
AR="gcc-ar"
NM="gcc-nm"
RANLIB="gcc-ranlib"
</syntaxhighlight>

<syntaxhighlight lang="bash">
CC="gcc"
CXX="g++"
CFLAGS="-fno-lto -march=native -O2 -pipe"
CXXFLAGS="${CFLAGS}"
AR="ar"
NM="nm"
RANLIB="ranlib"
</syntaxhighlight>

Basically, copy over your current working GCC config from your make.conf in the event we need to use it as a fallback. If you choose to use LLVM's implementation of AR, NM, and RANLIB as detailed later in the article, be sure to set them back to the GNU versions for your GCC fallback environments as shown in the above example. If you choose not to, you can ignore the AR, NM, and RANLIB variables. If you want to use link-time optimization it's a good idea to have two separate environments like the above examples.

In the event you have to use the GCC fallback environment(s) set the appropriate flags in the /etc/portage/package.env file.

app-foo/bar compiler-gcc-flto        #compiled using GCC with link-time optimization since package bar compiles using flto
app-bar/baz compiler-gcc-fno-lto     #compiled using GCC with no link-time optimization since package baz fails using flto

Alternatively, if you have app-portage/flaggie installed, the /etc/portage/package.env file could be modified using:

Testing against GCC

Those wishing to rebuild the system's @world set with clang should first enable tests via FEATURES=test in make.conf. Note that many developers do not actually care about their own tests, and as a result there may be many false positives. Because of this, it is wise to compare the package test results against GCC build tests. Some packages that fail with clang will also fail with gcc.

Clang environments

Now that we've set up a safe fallback we can proceed to enable the usage of Clang in Gentoo. There are two ways to do this: System wide using /etc/portage/make.conf or via environmental variables like the one(s) we created for the GCC fallback.

We'll use the same process as we did earlier in the article for setting up GCC fallbacks.

<syntaxhighlight lang="bash">
CC="clang"
CXX="clang++"
</syntaxhighlight>

You can now use Clang on a per package basis by invoking the compiler-clang environmental variable you created.

app-foo/bar compiler-clang
app-bar/baz compiler-clang

Of course, that's about as basic as it gets. You can create more complex environments if you wish. It's a good idea to have a variable for link-time optimized Clang and one without if you want to take advantage of LTO as detailed later in the article. Simply create two environments, one with the flag -flto and one with -fno-lto and apply each to packages as needed.

<syntaxhighlight lang="bash">
CC="clang"                            
CXX="clang++"                         
CFLAGS="${CFLAGS} -flto"              
CXXFLAGS="${CXXFLAGS} -flto"          
LDFLAGS="-Wl,-O2 -Wl,--as-needed"     #please use whichever optimization level you're comfortable with
</syntaxhighlight>

<syntaxhighlight lang="bash">
CC="clang"                            
CXX="clang++"                         
CFLAGS="${CFLAGS} -fno-lto"              
CXXFLAGS="${CXXFLAGS} -fno-lto"          
</syntaxhighlight>

You can now use Clang with or without link-time optimization on a per package basis by invoking the proper environmental you created. Please see "Using link-time optimizations with Clang" for a detailed guide.

app-foo/bar compiler-clang-flto      #compiled using Clang with link-time optimization since package bar compiles using flto
app-bar/baz compiler-clang-fno-lto   #compiled using Clang with no link-time optimization since package baz fails using flto

Global configuration via make.conf

When attempting to use Clang system wide the system absolutely must have a GCC fallback! This cannot be stressed enough as the system will not be able to compile everything using Clang at the moment, such as the GCC compiler. Configuring Gentoo to use Clang system wide is simple. Change the CC and CXX variables in /etc/portage/make.conf to reference the Clang equivalents. No further configuration is necessary.

<syntaxhighlight lang="bash">
CC="clang"
CXX="clang++"
</syntaxhighlight>

Packages that must use GCC for compiling can be handled with one of the fallback environments created earlier.

Bootstrapping the clang toolchain

Mixing clang and its toolchain / libraries with the gcc toolchain / libraries (especially the linker) will often lead to issues like linker errors during emerge. To prevent this, the clang toolchain is built first with gcc and then with itself to get a self-providing compiler.

Prepare the environment for the clang toolchain (see above), e.g.

CC="clang"
CXX="clang++"
CXXFLAGS="-stdlib=libc++ ${CFLAGS}"
LDFLAGS="-fuse-ld=lld -rtlib=compiler-rt -unwindlib=libunwind"

This example replaces not only the compiler but also the GNU linker ld with the llvm linker lld. It is a drop-in replacement, but about twice as fast as the GNU gold linker.

Set USE flags default-compiler-rt default-libcxx default-lld for clang. Then emerge clang llvm libcxx libcxxabi compiler-rt llvm-libunwind lld with the default gcc environment:

Note that sys-libs/llvm-libunwind deals with linking issues that sys-libs/libunwind has, so it is preferred to use and replaces the non-llvm libunwind package if installed (it builds with -lgcc_s to resolve issues with __register_frame / __deregister_frame undefined symbols).

Enable the clang environment for these packages now:

sys-devel/llvm compiler-clang
sys-libs/libcxx compiler-clang
sys-libs/libcxxabi compiler-clang
sys-libs/compiler-rt compiler-clang
sys-libs/compiler-rt-sanitizers compiler-clang
sys-libs/llvm-libunwind compiler-clang
sys-devel/lld compiler-clang
sys-devel/clang compiler-clang

Repeat the emerge step with the new environment. The toolchain will now be rebuild with itself instead of gcc.

It may be necessary to recompile all packages that depend on llvm directly the same way, like media-libs/mesa which uses llvm for graphic cards drivers. Otherwise gentoo keeps back some compatibility binaries to keep the packages from breaking immediately, but if this is forgotten, the packages may break on the next update. Just add them to package.env with compiler-clang environment and emerge them.

You are now free to use clang with other packages, like e.g. chromium. If these packages depend on further libraries (like dev-libs/re2 for chromium, these libraries must be built with clang as well, otherwise the linker will exit with undefined symbols).

Link-time optimizations with Clang

The link-time optimization feature defers optimizing the resulting executables to linking phase. This can result in better optimization of packages but isn't standard behavior in Gentoo yet. In order to use LTO we must first install LLVM with the Gold linker plugin so Clang can pass the -plugin flag to the linker.

sys-devel/llvm gold

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] sys-devel/llvm-7.1.0:7/7.1::gentoo  USE="gold* libffi ncurses xml -debug -doc -exegesis -libedit -test -xar" ABI_X86="(64) -32 (-x32)" LLVM_TARGETS="AMDGPU BPF NVPTX (X86) -AArch64 -ARM -Hexagon -Lanai -MSP430 -Mips -PowerPC -Sparc -SystemZ -XCore" 0 KiB
[ebuild  N     ] sys-devel/llvmgold-7::gentoo  0 KiB

Total: 2 packages (1 new, 1 reinstall), Size of downloads: 0 KiB

Would you like to merge these packages? [Yes/No]

/usr/bin/ld: unrecognized option '-plugin'
/usr/bin/ld: use the --help option for usage information

* Setting default linker to ld.gold for x86_64-pc-linux-gnu-2.25.1 ...

/usr/bin/ld: -plugin: missing argument
/usr/bin/ld: use the --help option for usage information

* Setting default linker to ld.bfd for x86_64-pc-linux-gnu-2.25.1 ...

LTO

<syntaxhighlight lang="bash">
CC="clang"                            
CXX="clang++"                         
CFLAGS="${CFLAGS} -flto"              
CXXFLAGS="${CXXFLAGS} -flto"          
LDFLAGS="-Wl,-O2 -Wl,--as-needed"     #please use whichever optimization level you're comfortable with
</syntaxhighlight>

<syntaxhighlight lang="bash">
CC="clang"                            
CXX="clang++"                         
CFLAGS="${CFLAGS} -flto"              
CXXFLAGS="${CXXFLAGS} -flto"          
LDFLAGS="-Wl,-O2 -Wl,--as-needed"    #please use whichever optimization level you're comfortable with
AR="llvm-ar"
NM="llvm-nm"
RANLIB="llvm-ranlib"
</syntaxhighlight>

app-foo/bar compiler-clang-flto
app-bar/baz compiler-clang-flto

<syntaxhighlight lang="bash">
CC="clang"                            
CXX="clang++"                         
CFLAGS="${CFLAGS} -flto"              
CXXFLAGS="${CXXFLAGS} -flto"          
LDFLAGS="-Wl,-O2 -Wl,--as-needed"    #please use whichever optimization level you're comfortable with
AR="llvm-ar"
NM="llvm-nm"
RANLIB="llvm-ranlib"
</syntaxhighlight>

app-foo/bar compiler-clang-fno-lto   #compiled using Clang with no link-time optimization since package bar fails using flto
app-bar/baz compiler-gcc-fno-lto     #compiled using GCC with no link-time optimization since package bar fails using flto
app-baz/foo compiler-gcc-flto        #compiled using GCC with link-time optimization since package foo compiles using flto

ThinLTO starting with Clang 3.9

As of Clang 3.9, a new feature called ThinLTO has been introduced. In a nutshell, ThinLTO gives you the performance of full LTO, but with the compile times of no LTO.^[3] To invoke the command, simply change -flto (which defaults to full when invoked) to -flto=thin in your configuration files.

The Gold linker is still required for this so make sure you've followed the steps above to set it as the default linker.

<syntaxhighlight lang="bash">
CC="clang"                            
CXX="clang++"                         
CFLAGS="${CFLAGS} -flto=thin"              
CXXFLAGS="${CXXFLAGS} -flto=thin"          
LDFLAGS="-Wl,-O2 -Wl,--as-needed"    #please use whichever optimization level you're comfortable with
</syntaxhighlight>

distcc

In order to use clang on a distcc client, additional symlinks have to be created in /usr/lib*/distcc/bin:

ccache

Automatic with `>=ccache-3.9-r3` when Clang is emerged.

Compile errors when using Clang with -flto

If the packages you're installing are failing, check your logs. Often times packages with errors like the following will need to disable LTO by invoking the compiler-clang-fno-lto environment.

/usr/bin/x86_64-pc-linux-gnu-ld: error: version.o:1:3: invalid character
/usr/bin/x86_64-pc-linux-gnu-ld: error: version.o:1:3: syntax error, unexpected $end
/usr/bin/x86_64-pc-linux-gnu-ld: error: version.o: not an object or archive

You will also most likely see this error in every LTO failure case.

x86_64-pc-linux-gnu-clang-3.8: error: linker command failed with exit code 1 (use -v to see invocation)

Simply add the failing package to your /etc/portage/package.env. In this case it's sys-apps/less, so we'll apply the proper override.

sys-apps/less compiler-clang-fno-lto   #compiled using Clang with no link-time optimization since package less fails using flto

Sometimes a package will fail to compile even when disabling LTO because it requires another package which was compiled using -flto and works incorrectly. You may see an error like this:

/usr/lib64/libatomic_ops.a: error adding symbols: Archive has no index; run ranlib to add one

In this case libatomic_ops is causing boehm-gc to fail compiling. Recompile the program causing the failure using your -fno-lto environment and then recompile the new program. In this case, boehm-gc fails when using LTO, so we'll add both of them to our /etc/portage/package.env file and assign the environments using the -fno-lto flags.

dev-libs/boehm-gc		compiler-clang-fno-lto
dev-libs/libatomic_ops		compiler-clang-fno-lto

Use of GNU extensions without proper -std=

Some packages tend to use GNU extensions in their code without specifying -std= appropriately. GCC allows that usage, yet Clang disables some of more specific GNU extensions by default.

If a particular package relies on such extensions being available, you will need to append the correct -std= flag to it:

• -std=gnu89 for C89/C90 with GNU extensions,
• -std=gnu99 for C99 with GNU extensions,
• -std=gnu++98 for C++:1998 with GNU extensions.

A common symptom of this problem are multiple definitions of inline functions like this:

/usr/bin/x86_64-pc-linux-gnu-ld: error: ../mpi/.libs/libmpi.a(mpi-bit.o): multiple definition of '_gcry_mpih_add'
/usr/bin/x86_64-pc-linux-gnu-ld: ../mpi/.libs/libmpi.a(mpi-add.o): previous definition here
/usr/bin/x86_64-pc-linux-gnu-ld: error: ../mpi/.libs/libmpi.a(mpi-bit.o): multiple definition of '_gcry_mpih_add_1'
/usr/bin/x86_64-pc-linux-gnu-ld: ../mpi/.libs/libmpi.a(mpi-add.o): previous definition here

This is because Clang uses C99 inline rules by default which do not work with gnu89 code. To work around it, you most likely have to pass -std=gnu89 or set one of your environmental overrides to use GCC to compile the failing package if passing the right -std= flag doesn't work.