Function shutil.copy opens source file with open('rb') and destination file with open('wb') and then proceeds to copy the content. If you create child process from the same python process during that time, that process will inherit open file handles. This could lead to situation when parent process tries to delete the file, but child process prevents that by holding an open handle to the file (Windows XP, Windows 7). This is not expected side effect for copy file operation. Win32's native CopyFile API call doesn't leak file handles to child processes. Python's open function behavior is defined by platform's implementation, which translates C function fopen with 'rb' and 'wb' modes to CRT open call without O_NOINHERIT flag, which creates inheritable Win32 HANDLEs. Possible fix would be to add 'N' to mode string in calls to open function in shutil.copy.
Unless I'm misunderstanding something, this has already been fixed in python3, where subprocess closes any open files (other than stdin/stdout/stderr) by default. In Python2.7 (and possibly earlier, I haven't checked) you can get the same effect by specifying close_fds=True in the Popen call. I don't see any reason why shutil should take any special action in this case, this is a general problem with open file descriptors and subprocess.
This problem also affects processes started by multiprocessing (although it is fixed in http://hg.python.org/sandbox/sbt#spawn). As far as I am concerned O_NOINHERIT should really have been applied by default in Python 3. You only get inheritable fds on Windows if you use the msvcrt's spawn*() functions. But in Python 3 os.spawn*() is implemented on top of subprocess so the fds are never inherited (even if the underlying handles are).
See also the PEP 433 which proposes to disable file descriptor inherance by default. 2013/4/5 Richard Oudkerk <report@bugs.python.org>: > > Richard Oudkerk added the comment: > > This problem also affects processes started by multiprocessing (although it is fixed in http://hg.python.org/sandbox/sbt#spawn). > > As far as I am concerned O_NOINHERIT should really have been applied by default in Python 3. You only get inheritable fds on Windows if you use the msvcrt's spawn*() functions. But in Python 3 os.spawn*() is implemented on top of subprocess so the fds are never inherited (even if the underlying handles are). > > ---------- > > _______________________________________ > Python tracker <report@bugs.python.org> > <http://bugs.python.org/issue17634> > _______________________________________
The issue has been fixed in Python 3.4 with the PEP 446. I close this issue. If you would like to enhance shutil.copyfile() by using CopyFile() on Windows, please open a new issue.
