Increase in orc jit's peak memory usage after upgrading from llvm-14 to llvm-17 (original) (raw)

Hi all,

We recently did an orc-jit upgrade from llvm-14 to llvm-17 and started noticing a higher memory usage. Using Windows Performance Recorder to track all VAlloc, we notice a 120% increase in peak memory usage going from 762MB to 1681MB.

Digging a bit deeper into the stack traces that lead up to this memory peak, I suspect the increase might be tied to the switch to using JITLink by default (Re-apply "[ORC][LLJIT] Use JITLink by default on ELF/x86-64." with fi… · llvm/llvm-project@b92839c · GitHub).

For instance, in llvm 14, a stack trace at peak looked something like this:

llvm::orc::MaterializationTask::run/llvm::orc::BasicIRLayerMaterializationUnit::materialize/llvm::orc::IRTransformLayer::emit/llvm::orc::IRTransformLayer::emit/llvm::orc::IRCompileLayer::emit/llvm::orc::ObjectTransformLayer::emit/llvm::orc::RTDyldObjectLinkingLayer::emit/llvm::jitLinkForORC/llvm::RuntimeDyld::loadObject/llvm::RuntimeDyldELF::loadObject/llvm::RuntimeDyldImpl::loadObjectImpl/llvm::RuntimeDyldImpl::findOrEmitSection/llvm::RuntimeDyldImpl::emitSection/llvm::SectionMemoryManager::allocateSection/llvm::`anonymous namespace’::DefaultMMapper::allocateMappedMemory/llvm::sys::Memory::allocateMappedMemory/KernelBase.dll!VirtualAlloc/

And in llvm 17, just the computeBlockNonLocalDeps part is hogging 1000MB out of 1681MB at peak:

llvm::orc::MaterializationTask::run/
llvm::orc::BasicIRLayerMaterializationUnit::materialize/
llvm::orc::IRTransformLayer::emit/
llvm::orc::IRTransformLayer::emit/
llvm::orc::IRCompileLayer::emit/
llvm::orc::ObjectTransformLayer::emit/
llvm::orc::ObjectLinkingLayer::emit/
llvm::jitlink::link/
llvm::jitlink::link_ELF/
llvm::jitlink::link_ELF_x86_64/
llvm::jitlink::JITLinkerBase::linkPhase1/
llvm::detail::UniqueFunctionBase<llvm::Error,llvm::jitlink::LinkGraph &>::CallImpl<llvm::orc::ObjectLinkingLayerJITLinkContext::modifyPassConfig'::2’::<lambda_2> >/
llvm::orc::ObjectLinkingLayerJITLinkContext::computeNamedSymbolDependencies/
llvm::orc::ObjectLinkingLayerJITLinkContext::computeBlockNonLocalDeps/
llvm::DenseMap<llvm::Instruction *,llvm::detail::DenseSetEmpty,llvm::DenseMapInfo<llvm::Instruction *,void>,llvm::detail::DenseSetPair<llvm::Instruction *> >::grow/
operator new/
ucrtbase.dll!_aligned_malloc/
ucrtbase.dll!_malloc_base/

Does anyone have insights into why the memory usage for orc’s jit has spiked so much? And I wonder if there an opportunity to optimize this?

Thanks