Difficulty building clang from 20.1.6 for armv8-unknown-unknown-eabihf-elf (original) (raw)

I’m trying to build an LLVM-based toolchain for an embedded device (Cortex-M33 based, so ARMv8-M) on which I’m bringing up a new operating system (referred to as “Fuji” below) using the LLVM 20.1.6 release.

I can build a compiler, linker, and debugger using the following invocation on either macOS 15.5 (with Xcode 16.5) or Fedora 42 (with Fedora’s clang, plus the latest cmake and ninja):

$ mkdir build
$ cd build
$ cmake ../llvm \
        -G Ninja \
          -DCMAKE_C_COMPILER="clang" \
          -DCMAKE_CXX_COMPILER="clang++" \
          -DCMAKE_BUILD_TYPE="Release" \
          -DCMAKE_INSTALL_PREFIX="/opt/Fuji/Tools" \
          -DLLVM_DEFAULT_TARGET_TRIPLE="armv8-unknown-unknown-eabihf-elf" \
          -DLLVM_ENABLE_PROJECTS="clang;lld;lldb" \
          -DLLVM_TARGETS_TO_BUILD="ARM" \
          -DLLVM_TARGET_ARCH="ARM" \
          -DCLANG_DEFAULT_OBJCOPY="llvm-objcopy" \
          -DCLANG_DEFAULT_LINKER="lld" \
          -DLLVM_INSTALL_TOOLCHAIN_ONLY="ON" \
          -DDEFAULT_SYSROOT="/opt/Fuji/SDKs/Fuji.sdk" \
          -DLLVM_INCLUDE_EXAMPLES="OFF" \
          -DLLVM_INCLUDE_TESTS="OFF" \
          -DLLVM_INSTALL_BINUTILS_SYMLINKS="ON"

However, I assume I should also build compiler-rt, since this is a new platform that has no other runtime (yet—I’m hoping to use LLVM libc too). But adding the following into the above invocation fails:

          -DLLVM_ENABLE_RUNTIMES="compiler-rt" \
          -DCLANG_DEFAULT_RTLIB="compiler-rt" \

Here’s the failure on macOS:

CMake Error at /Users/cmh/Projects/LLVM/llvm-project/compiler-rt/CMakeLists.txt:500 (message):
  -g is not supported by host compiler

And here’s the failure in a Fedora 42 container using Fedora’s clang, with a locally-built CMake 4.0.2 and Ninja 1.12.1:

-- Configuring done (1.6s)
CMake Error at /opt/Fuji/Source/llvm-project-20.1.6.src/compiler-rt/cmake/Modules/AddCompilerRT.cmake:347 (add_custom_command):
  Attempt to add a custom rule to output

    /opt/Fuji/Source/llvm-project-20.1.6.build/lib/clang/20/lib/arm-unknown-unknown-eabihf-elf/clang_rt.crtbegin.o.rule

  which already has a custom rule.
Call Stack (most recent call first):
  CMakeLists.txt:992 (add_compiler_rt_runtime)


CMake Error at /opt/Fuji/Source/llvm-project-20.1.6.src/compiler-rt/cmake/Modules/AddCompilerRT.cmake:347 (add_custom_command):
  Attempt to add a custom rule to output

    /opt/Fuji/Source/llvm-project-20.1.6.build/lib/clang/20/lib/arm-unknown-unknown-eabihf-elf/clang_rt.crtend.o.rule

  which already has a custom rule.
Call Stack (most recent call first):
  CMakeLists.txt:998 (add_compiler_rt_runtime)

What am I doing wrong here?

Hi, you may want to have a look at arm-toolchain/arm-software/embedded/README.md at arm-software · arm/arm-toolchain · GitHub to either use the binaries or the build scripts to produce a toolchain to target Cortex-M33.

That looks useful, but I’d really prefer to use as close to an LLVM mainline release as possible for what I’m doing.

I just want to build a compiler-rt alongside clang and lld and lldb for my (now) armv8m.main-unknown-unknown-llvm-elf toolchain. It doesn’t seem like that’s really all that big an ask.

I don’t think that the build system can do a single CMake invocation that builds the tools for the host system on an hosted platform yet cross-compiles the runtimes for an embedded environment. In particular the host build will use the C headers from the compiler you are using on your host OS, but you’ll need to use the C headers from the embedded C-library you are using.

In the past I’ve split the build up into separate CMake invocations. First build the tools with CMake pointing at the llvm directory. Then build compiler-rt separately with CMake pointing at the compiler-rt directory. I would pass in the include directories for the C-library I was using, and make sure to tell the build system I was building a static library only.

You can see part of this in the runtimes build arm-toolchain/arm-software/embedded/arm-runtimes/CMakeLists.txt at arm-software · arm/arm-toolchain · GitHub

There are some good tutorials and articles at recent LLVM dev meetings. For example:
https://llvm.org/devmtg/2023-10/slides/tutorials/Hosek-UnderstandingtheLLVMbuild.pdf
https://llvm.org/devmtg/2024-10/slides/techtalk/Hosek-ModernEmbeddedDevelopment-with-LLVM.pdf

You may also want to take a look at the Google Pigweed Toolchain that I think can be built from the Fuchsia toolchain in https://cs.opensource.google/fuchsia/fuchsia/+/main:scripts/clang/

Ah, OK. That makes sense, for some reason I thought it was actually possible to build for both the host and target at the same time. I’ll give successive builds a try, thanks!

I think what confused me was this from the page Building LLVM with CMake on LLVM_ENABLE_RUNTIMES:

Build libc++, libc++abi, libunwind or compiler-rt using the just-built compiler. This is the correct way to build runtimes when putting together a toolchain. It will build the builtins separately from the other runtimes to preserve correct dependency ordering. If you want to build the runtimes using a system compiler, see the libc++ documentation.

I took this to imply that if I was building to run on the host—but prefer a different default target triple than the host—that the runtimes would be built for that target triple using the just-built compiler.

OK, so I’m still having an incredible amount of trouble with this.

Here’s exactly what I want to do, on both Linux and macOS:

  1. Build clang, lld, and lldb LLVM 20.1.6 with a default target triple of armv8.main-none-none-eabihf-elf, a default install prefix of /opt/Fuji/Tools, a default sysroot of /opt/Fuji/SDKs/Fuji.sdk, and only “toolchain” tools installed into subpaths of the install prefix.
  2. Use those tools to build compiler-rt, libcxx, libcxxabi, libunwind for the above target triple with their install prefix set to a place where they will be picked up automatically by the tools built in step 1 when invoked—whether that’s either /opt/Fuji/Tools or /opt/Fuji/SDKs/Fuji.sdk doesn’t really matter to me.

That’s it. It seems like this would be a very common case for setting up a cross-development toolchain, but I have had an incredibly difficult time coming up with a series of CMake invocations that will actually cause it to happen. This is with a stock LLVM 20.1.6 distribution too, nothing leading-edge.

Rolling your own embedded toolchain isn’t that common, and can be frustrating as lots of things need to line up for it to work.It is one of the reasons we’re pointing you at existing toolchains where that’s already been done.

The first thing to work out which Driver your triple of armv8.main-none-none-eabihf-elf is selecting. My suspicion is that it will be the fallback GCC driver rather than the bare metal driver. I would expect a triple of armv8.main-unknown-none-eabihf or armv8.main-none-eabihf with an unknown vendor and an eabihf environment, it is possible that the -elf suffix may be throwing off the Driver selection code.

I’d personally see what I can build after step 1, making heavy use of the -v option to see what include and library paths the compiler is using. If the driver isn’t doing the right thing you may need to supply your own. Clang config files are good for this.

Once you know what Driver (ToolChain) clang is using, I often find it useful to figure out what Clang is doing by looking through it llvm-project/clang/lib/Driver/ToolChains at main · llvm/llvm-project · GitHub

OK, I guess the way to use clang for my new platform then is to build the compiler “generically” and pass a giant pile of parameters to get it to do the right thing.

It’d be nice to have specific documentation on “here’s how to add a new platform to LLVM/clang/runtimes” so just passing the right target tuple will ensure the right thing happens.

I’ll take my runtime build difficulties to a thread in the runtimes subforum.