Lazy compilation is not thread-safe (original) (raw)

Extended Description

Lazy compilation is implemented roughly as:

call @​compilationCallback

define @​compilationCallback() {
...
store %new_function_address (%return_address-4)
}

which turns the "call @​compilationCallback" into a "call @​real_function". Unfortunately, while that store into the return address is running, another thread could be executing the call. At least on the x86, there are no guarantees about what can happen in this case. (Data reads and writes are defined, but not instruction execution.) It's entirely possible that the store won't be seen as atomic and will result in a wild jump.

I see three ways to fix this:

1. Instead of codegening a direct call instruction, generate something like:
   %target = atomic load @​callsite_target_address
   call %target
   and then at the end of compilation atomically store the real function back to @​callsite_target_address. This causes every callsite to be an indirect call, which is likely to slow things down a lot, so there should be some way to mark the functions where it's used. On platforms that don't even have atomic pointer writes, this could be implemented by acquiring a lock around every lazily-compilable call.
2. Assert that lazy compilation is only used in a single-threaded program. This allows us to delete the JIT lock.
3. Delete lazy compilation. (which would simplify the JIT a fair amount)