Message 135095 - Python tracker (original) (raw)

Yes, we would need to keep track of the thread id and process id inside the lock. We also need a global variable of the main thread id after fork, and a per-lock "taken" flag.

Synopsis:

   def _reinit_if_needed(self):        # Call this before each acquire() or release()        if self.pid != getpid():            sem_init(self.sem, 0, 1)            if self.taken:                if self.tid == main_thread_id_after_fork:                    # Lock was taken in forked thread, re-take it                    sem_wait(self.sem)                else:                    # It's now released                    self.taken = False            self.pid = getpid()            self.tid = current_thread_id()

A couple remarks:

• with linuxthreads, different threads within the same process have the same PID - it may be true for other implementations - so this would lead to spurious reinitializations
• what's current_thread_id ? If it's thread_get_ident (pthread_self), since TID is not guaranteed to be inherited across fork, this won't work
• calling getpid at every acquire/release is expensive, even though it's a trivial syscall (it'll have to measured though)
• imagine the following happens:

P1

lock.acquire() fork() -> P2 start_new_thread T2 T1 T2 lock.acquire()

The acquisition of lock by T2 will cause lock's reinitialization: what happens to the lock wait queue ? who owns the lock ? That why I don't think we can delay the reinitialization of locks, but I could be wrong.

Well, I fail to understand how that idiom can help us. We're not a self-contained application, we're a whole programming language. Calling fork() only when no lock is held is unworkable (for example, we use locks around buffered I/O objects).

Yes, but in that case, you don't have to reacquire the locks after fork. In the deadlock you experienced above, the thread that forked wasn't the one in the I/O code, so the corresponding lock can be re-initialized anyway, since the thread in the I/O code at that time won't exist after fork. And it's true with every lock in the library code: they're only held in short critical sections (typically acquired when entering a function and released when leaving), and since it's not the threads inside those libraries that fork, all those locks can simply be reinitialized on fork, without having the reacquire them.