Using same semaphore in two different python programs (original) (raw)

February 27, 2026, 8:28am 1

Is there any way to share a common semaphore between different python programs. Currently multiprocessing module allows semaphore to be used between processes forked/spawned from current process. In C on using POSIX semaphores, we can have a producer and consumer as two different programs and share a common semaphore. similarly in python can two different .py files share a common semaphore using the built-in python modules?

Rosuav (Chris Angelico) February 27, 2026, 8:46am 2

Yes, but it has to be something outside of both. For example, it’s very common to use a filesystem lock to govern something, or have the two processes connect to a single manager that makes the decisions. It depends on your design as to what’s best.

zuhu2195 (Mohammed Zuhaib) February 27, 2026, 8:51am 3

Can’t it be something like the producer program creates the semaphore and then consumer and producer operate on the same (like in C using sem_open)? When a single manager makes decisions, the semaphore is in control of manager and neither programs will be able to use it to make decisions right?

JamesParrott (James Parrott) February 27, 2026, 8:57am 4

What would then happen if two different processes run the same.py file (i.e. twice)?
[Edit] If such a thing existed, it would just bump the counter on the Semaphore an extra time.

Rosuav (Chris Angelico) February 27, 2026, 8:58am 5

A single manager that issues tasks to others? That would work fine; tasks can be issued through a multiprocess queue of some form, and the other processes will simply draw from that.

There are some multiprocess semaphores. The most common ones are found on the file system (at least on Unix - maybe not on Windows) or sockets/pipes.

peterc (peter) February 27, 2026, 9:00am 6

You can use flag files. Not sure if that’s applicable to your situation, and it’s not perfectly robust. But it is a way to have variables that are “global” on the level of “being accessible to all programs that are running”. And they can be changed atomically.

zuhu2195 (Mohammed Zuhaib) February 27, 2026, 9:20am 7

Yes, but even that would be same as adding a new producer or consumer to the mix and the results should not be much different right? Applying the same analogy to a producer-consumer problem in C, we can run the either producer or consumer more than once, the user needs to take care of the logic.

I found another thread which discussed the same issue,

can python multiprocessing do semaphores between “unrelated” processes? - Stack Overflow

So, seems like for semaphores we cannot do this in python(unlike in C). Other ways like flag files pointed out by @peterc or ones pointed by @Rosuav can work but it seems to be absent for semaphores as of now.

Third party extension authors have tried to wrap POSIX Semaphores for use in Python. I don’t know which the best choice is (nor how suitable they actually are) but, e.g.:

barry-scott (Barry Scott) February 27, 2026, 8:39pm 9

You can use named semaphrores on Unix and Windows, APIs are not the same, but it’s doable.

If you need example code I can post a link to one of my scripts that does this. Let me know.

zuhu2195 (Mohammed Zuhaib) February 27, 2026, 11:20pm 10

Yes, please do provide the link to the script.

barry-scott (Barry Scott) February 27, 2026, 11:32pm 11

See the different _sendCommand functions that use es h OS method to talk the Barry’s emacs server process.

Code is here BarrysEmacs/Editor/PyQt6/be_client.py at master · barry-scott/BarrysEmacs · GitHub

The server code is C++, I can link to that if you need it for reference.

zuhu2195 (Mohammed Zuhaib) February 28, 2026, 7:50am 12

Thank you Barry, yes please do share the server code.

barry-scott (Barry Scott) March 2, 2026, 9:57am 13

zuhu2195 (Mohammed Zuhaib) March 2, 2026, 10:44am 14