[Python-Dev] shm + Win32 + docs (was: Adding library modules to the core) (original) (raw)
Vladimir Marangozov Vladimir.Marangozov@inrialpes.fr
Thu, 17 Aug 2000 04:12:18 +0200 (CEST)
- Previous message: [Python-Dev] Adding library modules to the core
- Next message: [Python-Dev] Adding library modules to the core
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Eric S. Raymond wrote:
Vladimir, I suggest that the most useful thing you could do to advance the process at this point would be to document shm in core-library style.
Eric, I'm presently suffering from chronic lack of time (as you probably do too) so if you could write the docs for me and take all associated credits for them, please do so (shouldn't be that hard, after all -- the web page and the comments are self-explanatory :-). I'm willing to "unblock" you on this, but I can hardly make the time for it -- it's low-priority on my dynamic task schedule. :(
I'd also love to assemble the win32 bits on the matter (what's in win32event for the semaphore interface + my Windows book) to add shm and sem for Windows and rewrite the interface, but I have no idea on when this could happen.
I will disappear from the face of the World sometime soon and it's unclear on when I'll be able to reappear (nor how soon I'll disappear, btw) So, be aware of that. I hope to be back again before 2.1 so if we can wrap up a Unix + win32 shm, that would be much appreciated!
At the moment, core Python has nothing (with the weak and nonportable exception of open(..., OEXCL)) that can do semaphores properly. Thus shm would address a real gap in the language.
Indeed. This is currently being discussed on the french Python list, where Richard Gruet (rgruet@ina.fr) posted the following code for inter-process locks: glock.py
I don't have the time to look at it in detail, just relaying here for food and meditation :-)
-- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252
------------------------------[ glock.py ]---------------------------- #!/usr/bin/env python #----------------------------------------------------------------------------
glock.py: Global mutex
Prerequisites:
- Python 1.5.2 or newer (www.python.org)
- On windows: win32 extensions installed
(http://www.python.org/windows/win32all/win32all.exe)
- OS: Unix, Windows.
History:
-22 Jan 2000 (R.Gruet): creation
Limitations:
TODO:
#---------------------------------------------------------------------------- ''' This module defines the class GlobalLock that implements a global (inter-process) mutex that works on Windows and Unix, using file-locking on Unix (I also tried this approach on Windows but got some tricky problems so I ended using Win32 Mutex).. See class GlobalLock for more details. ''' version = 1,0,2 author = ('Richard Gruet', 'rgruet@ina.fr')
Imports:
import sys, string, os
System-dependent imports for locking implementation:
_windows = (sys.platform == 'win32')
if _windows: try: import win32event, win32api, pywintypes except ImportError: sys.stderr.write('The win32 extensions need to be installed!') else: # assume Unix try: import fcntl except ImportError: sys.stderr.write("On what kind of OS am I ? (Mac?) I should be on " "Unix but can't import fcntl.\n") raise import threading
Exceptions :
----------
class GlobalLockError(Exception): ''' Error raised by the glock module. ''' pass
class NotOwner(GlobalLockError): ''' Attempt to release somebody else's lock. ''' pass
Constants
---------:
true=-1 false=0
#---------------------------------------------------------------------------- class GlobalLock: #---------------------------------------------------------------------------- ''' A global mutex.
*Specification:
-------------
The lock must act as a global mutex, ie block between different
candidate processus, but ALSO between different candidate
threads of the same process.
It must NOT block in case of recursive lock request issued by
the SAME thread.
Extraneous unlocks should be ideally harmless.
*Implementation:
--------------
In Python there is no portable global lock AFAIK.
There is only a LOCAL/ in-process Lock mechanism
(threading.RLock), so we have to implement our own solution.
Unix: use fcntl.flock(). Recursive calls OK. Different process OK.
But <> threads, same process don't block so we have to
use an extra threading.RLock to fix that point.
Win: We use WIN32 mutex from Python Win32 extensions. Can't use
std module msvcrt.locking(), because global lock is OK, but
blocks also for 2 calls from the same thread!
'''
def __init__(self, fpath, lockInitially=false):
''' Creates (or opens) a global lock.
@param fpath Path of the file used as lock target. This is also
the global id of the lock. The file will be created
if non existent.
@param lockInitially if true locks initially.
'''
if _windows:
self.name = string.replace(fpath, '\\', '_')
self.mutex = win32event.CreateMutex(None, lockInitially, self.name)
else: # Unix
self.name = fpath
self.flock = open(fpath, 'w')
self.fdlock = self.flock.fileno()
self.threadLock = threading.RLock()
if lockInitially:
self.acquire()
def __del__(self):
#print '__del__ called' ##
try: self.release()
except: pass
if _windows:
win32api.CloseHandle(self.mutex)
else:
try: self.flock.close()
except: pass
def __repr__(self):
return '<Global lock @ %s>' % self.name
def acquire(self):
''' Locks. Suspends caller until done.
On windows an IOError is raised after ~10 sec if the lock
can't be acquired.
@exception GlobalLockError if lock can't be acquired (timeout)
'''
if _windows:
r = win32event.WaitForSingleObject(self.mutex, win32event.INFINITE)
if r == win32event.WAIT_FAILED:
raise GlobalLockError("Can't acquire mutex.")
else:
# Acquire 1st the global (inter-process) lock:
try:
fcntl.flock(self.fdlock, fcntl.LOCK_EX) # blocking
except IOError: #(errno 13: perm. denied,
# 36: Resource deadlock avoided)
raise GlobalLockError('Cannot acquire lock on "file" %s\n' %
self.name)
#print 'got file lock.' ##
# Then acquire the local (inter-thread) lock:
self.threadLock.acquire()
#print 'got thread lock.' ##
def release(self):
''' Unlocks. (caller must own the lock!)
@return The lock count.
@exception IOError if file lock can't be released
@exception NotOwner Attempt to release somebody else's lock.
'''
if _windows:
try:
win32event.ReleaseMutex(self.mutex)
except pywintypes.error, e:
errCode, fctName, errMsg = e.args
if errCode == 288:
raise NotOwner("Attempt to release somebody else's lock")
else:
raise GlobalLockError('%s: err#%d: %s' % (fctName, errCode,
errMsg))
else:
# Acquire 1st the local (inter-thread) lock:
try:
self.threadLock.release()
except AssertionError:
raise NotOwner("Attempt to release somebody else's lock")
# Then release the global (inter-process) lock:
try:
fcntl.flock(self.fdlock, fcntl.LOCK_UN)
except IOError: # (errno 13: permission denied)
raise GlobalLockError('Unlock of file "%s" failed\n' %
self.name)
#---------------------------------------------------------------------------- # M A I N #---------------------------------------------------------------------------- def main(): # unfortunately can't test inter-process lock here! lockName = 'myFirstLock' l = GlobalLock(lockName) if not _windows: assert os.path.exists(lockName) l.acquire() l.acquire() # rentrant lock, must not block l.release() l.release() if _windows: try: l.release() except NotOwner: pass else: raise Exception('should have raised a NotOwner exception')
# Check that <> threads of same process do block:
import threading, time
thread = threading.Thread(target=threadMain, args=(l,))
print 'main: locking...',
l.acquire()
print ' done.'
thread.start()
time.sleep(3)
print '\nmain: unlocking...',
l.release()
print ' done.'
time.sleep(0.1)
del l # to close file
print 'tests OK.'
def threadMain(lock): print 'thread started(%s).' % lock print 'thread: locking (should stay blocked for ~ 3 sec)...', lock.acquire() print 'thread: locking done.' print 'thread: unlocking...', lock.release() print ' done.' print 'thread ended.'
if name == "main": main()
- Previous message: [Python-Dev] Adding library modules to the core
- Next message: [Python-Dev] Adding library modules to the core
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]