[RFC] Move ORC executor support into top-level project (original) (raw)

September 4, 2024, 6:29am 1

Hi All,

Summary:

I think we should merge the LLVM OrcTargetProcess and compiler-rt ORC runtime libraries into a new top-level “orc-rt” project.

Background:

LLVM’s ORC JIT APIs support cross-process and cross-architecture execution of JIT’d code. Support code for out-of-process execution falls into two categories:

(1) Code that must be pre-linked into the executing process to bootstrap JIT’d code (e.g. memory managers, symbol resolvers, the JIT “executor-process-control server” code).

(2) Runtime support for advanced object format features for JIT’d code, e.g. static initializers, destructors, exceptions and thread-local-storage. This can be JIT-linked into the executor process by ORC.

Code for (1) currently lives in the LLVM OrcTargetProcess library, and code for (2) in the compiler-rt ORC runtime library.

Motivation

Moving this code into a new top-level project would address a few issues with our current setup:

  1. The bootstrap code could be built for all systems that the compiler targets, rather than just the host architecture. (OrcTargetProcess is currently only built for the LLVM host architecture).
  2. Compiler-rt developers wouldn’t have to explicitly opt out of building the ORC runtime (which many, if not most, compiler-rt devs do not use).
  3. JIT developers wouldn’t have to build all of compiler-rt (or explicitly opt out of all other compiler-rt libraries) in order to test the JIT.

Proposal

We should move the OrcTargetProcess and ORC runtime libraries into a new top-level “orc-rt” project inside llvm-project. We should also replace the llvm-jitlink-executor test tool with an orc-executor test tool in the new “orc-rt” project. This would allow us to implement out-of-process execution tests in “orc-rt” alongside the existing regression tests from the compiler-rt ORC runtime library.

We should duplicate some basic bootstrap support in LLVM: enough to provide the same level of functionality as MCJIT (this will allow MCJIT clients to move to ORC without building the new project). Use of more advanced features (including lazy compilation, thread local storage, out-of-process initializers / deinitializers, etc.), may in future require building the “orc-rt” project. Since this would only involve enabling one extra top-level LLVM project I think it’s a reasonable burden to place on JIT developers

What do you think?

I’ve started putting together a sketch of the new project in to serve as a basis for discussion / prototyping for anyone who wants to get involved:

vchuravy September 4, 2024, 12:57pm 2

From a Julia perspective, I am on board! I recently was thinking about a use-case where I don’t want the rest of LLVM, but just JITLink, and this would be very helpful!

Agree, that would be useful! However, I think Lang is proposing to just move out the ORC runtime implementation from OrcTargetProcess (e.g. debugger support symbols) and merge it with the orc-rt implementation (e.g. TLS, initializers) which lives in compiler-rt today. In the proposal here, ORC and JITLink would stay in LLVM.

lhames September 6, 2024, 1:04am 4

In the proposal here, ORC and JITLink would stay in LLVM.

That’s right. Only llvm-jitlink-executor would move to the new ORC-RT.

I do want to make ORC / JITLink usable without pulling in LLVM IR too (for clients who already have a compilation service and just want to accept raw objects from that) – that’ll reduce the baggage for using JITLink, but it’ll be a while before it happens (we’ll need to either break the dependence between ORC and Object, or Object and LLVM Core – any of those will take some work).

RVP September 6, 2024, 1:41am 5

+1 This is a good idea.

runtime libraries into a new top-level “orc-rt” project

Inside LLVM people probably know ORC as the JIT and won’t confuse it with other things that have the same name (like the debug-info format, this DSL, or all kinds of things that orchestrate sth). If it becomes a top-level project now, that might be different. Why not call it jit-rt instead or orcjit-rt? That seems more obvious to me.

lhames October 23, 2024, 9:45pm 7

It seems like people are basically on-board with this idea – that’s great!

I’ve posted a PR for an updated version of the new orc-rt project here: [ORC-RT] Initial check-in for a new, top-level ORC runtime project. by lhames · Pull Request #113499 · llvm/llvm-project · GitHub. This new version makes the top-level project an LLVM runtime rather than a standard project, but it is otherwise similar to the previous prototype.

In discussions at the LLVM Dev Meeting we’ve talked about using this project to also drive development of generic CMake build infrastructure for runtimes. In practice we expect this will mostly involve de-duplicating existing infrastructure used in libunwind / libcxx / libcxxabi. This does not affect the primary goal of the top-level ORC runtime project, but does mean that we’ll try to take extra care to make small incremental commits wherever we’re changing the build system (so that those changes are easy to understand in isolation from general ORC runtime work).

lhames October 23, 2024, 9:58pm 8

That’s a good point. “orcjit” is unfortunate, since ORC is already an acronym for “On Request Compilation”, but it would be very on-brand for an LLVM project to abandon its acronym. :wink:

I’m ok with the project going in as “orc-rt” for the initial commits – it will be a while before it’s ready for release. We should continue to talk about the name though. If we do decide to rename it to avoid confusion / collision then I prefer “orcjit” to plain “jit” – the latter seems even more likely to clash with other projects, especially since ORC has historically been used as a component in other JITs. We’ll also need to figure out what to do with the namespace (should it stay “orc_rt”, or should it be “orcjit_rt”?).

That’s a good point. “orcjit” is unfortunate, since ORC is already an acronym for “On Request Compilation”, but it would be very on-brand for an LLVM project to abandon its acronym. :wink:

Haha, yes it is! Didn’t that happen a long time ago?

➜ ninja -t query LLVMOrcJIT     
LLVMOrcJIT:
  input: phony
    lib/libLLVMOrcJIT.a
  outputs:

Lang has been helping us come up with a plan to migrate LLDB from MCJIT to ORC and this would help for that as well. With the current layout, using the ORC runtime would introduce a hard dependency on compiler-rt, which is something we’d like to avoid.

lhames October 28, 2024, 5:21pm 11

Does anyone have any thoughts on how a top-level ORC runtime project should be handled with regards to the new-top-level-project / incubator process:

https://llvm.org/docs/DeveloperPolicy.html#incubating-new-projects

We definitely want to add a README, docs, etc. Beyond that I’m not sure that the ORC runtime needs a lot of the infrastructure that we’d usually associate with a new top level project – e.g. a separate mailing list, domain, etc. I expect Discourse and the jit discord channel to be sufficient for discussion in the near term. The only thing I’m not sure about is where the docs would be surfaced, but I think it’d be fine to ask developers to just read the text docs for now.