| msg90821 - (view) |
Author: Timothee Besset (TTimo) |
Date: 2009-07-22 20:50 |
| shutil.copytree fails if there is a dangling symlink and symlink is set to False (which is the default). It will raise an exception when trying to get to the content of the symlink. Tested on Debian Etch amd64, python 2.5.2 File "/usr/lib/python2.5/shutil.py", line 138, in copytree raise Error, errors shutil.Error: [('/opt/daemons/gameslave/.#fabfile.py', '/tmp/tmphxInsp/gameslave/.#fabfile.py', "[Errno 2] No such file or directory: '/opt/daemons/gameslave/.#fabfile.py'")] $ ls -1l /opt/daemons/gameslave/.#fabfile.py lrwxrwxrwx 1 timo quakelive 20 Jul 22 14:32 /opt/daemons/gameslave/.#fabfile.py -> timo@localhost.12341 (the link is created by emacs - means file being edited I'm guessing) |
|
|
| msg103642 - (view) |
Author: Tarek Ziadé (tarek) *  |
Date: 2010-04-19 21:42 |
| Sorry, what's a "dangling symlink" ? can you provide steps to reproduce this ? I am moving the Python version to 2.6. If we are unable to reproduce it for Python > 2.5 we'll close it since we don't fix bugs for 2.5 anymore |
|
|
| msg103647 - (view) |
Author: Timothee Besset (TTimo) |
Date: 2010-04-19 22:16 |
| It's a symlink that points to a file that doesn't exist. There are many ways this can happen, in this particular case my text editor (emacs) seems to keep some metadata about which user, machine and process is editing a file. I tried to reproduce in 2.6 (Debian sid amd64) and I can confirm it still happens: timo@ttimozilla:~$ mkdir test timo@ttimozilla:~$ cd test timo@ttimozilla:~/test$ ln -s foo bar timo@ttimozilla:~/test$ ls -1l bar lrwxrwxrwx 1 timo timo 3 Apr 19 17:12 bar -> foo timo@ttimozilla:~/test$ ls -1l foo ls: cannot access foo: No such file or directory timo@ttimozilla:~/test$ python2.6 Python 2.6.5 (r265:79063, Mar 20 2010, 03:56:44) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import shutil >>> shutil.copytree( '../test', '../test2' ) Traceback (most recent call last): File "", line 1, in File "/usr/lib/python2.6/shutil.py", line 177, in copytree raise Error, errors shutil.Error: [('../test/bar', '../test2/bar', "[Errno 2] No such file or directory: '../test/bar'")] >>> |
|
|
| msg103655 - (view) |
Author: Tarek Ziadé (tarek) * |
Date: 2010-04-19 22:55 |
| ok. what behavior would you expect in that case ? I propose that copytree() ignores symlinks that are not linked to an existing file in the copy process, and display a warning in this case. We could add an option to transform this warning into an error that would stop the process. |
|
|
| msg103656 - (view) |
Author: Timothee Besset (TTimo) |
Date: 2010-04-19 23:00 |
| I am not sure what shutil does with symlinks already. At the very least it should not abort the operation. Ideally I feel it should create the same symlink pointing to a possibly missing file, since that's what '/bin/cp' does, and shutil.copytree is broadly understood as a drop in replacement.. |
|
|
| msg103657 - (view) |
Author: Tarek Ziadé (tarek) * |
Date: 2010-04-19 23:08 |
| > Ideally I feel it should create the same symlink pointing to a > possibly missing file, since that's what '/bin/cp' does, > and shutil.copytree is broadly understood as a drop in replacement.. That's what would happen if the symlink option is set to True. When False, copytree() is supposed to copy the file pointed by the symlink, so copying the symlink as a fallback in case the file doesn't exists seems wrong to me in case symlink=False |
|
|
| msg103681 - (view) |
Author: Tarek Ziadé (tarek) * |
Date: 2010-04-20 08:13 |
| Please ignore my last message. I've written a test for your error, and it turns out that the error is caught and just raised at the end of the process, meaning that everything else was copied before you get the exception. So shutil.copytree didn't fail. Now, the behavior is a little bit hard for this case, so I am going to add an option to ignore errors when a dangling symlink is found. |
|
|
| msg103684 - (view) |
Author: Tarek Ziadé (tarek) * |
Date: 2010-04-20 09:07 |
| Added in r80244. This new option willl be available in the next 3.2 release. Until 3.2 is released, what you can do is catch the errors and re-raise the ones that are not due to dangling symlinks: (not tested) >>> from shutil import copytree, Error >>> try: ... shutil.copytree('../test', '../test2') ... except Error, e: ... for src, dst, error in e.args[0]: ... if not os.path.islink(src): ... raise ... else: ... linkto = os.readlink(srcname) ... if os.path.exists(linkto): ... raise ... # dangling symlink found.. ignoring.. ... |
|
|
| msg103714 - (view) |
Author: Timothee Besset (TTimo) |
Date: 2010-04-20 14:19 |
| Good stuff. Didn't occur to me that the operation could have successfully completed before raising the exception |
|
|
| msg103715 - (view) |
Author: Tarek Ziadé (tarek) * |
Date: 2010-04-20 14:20 |
| Thanks for your work on Quake btw ;) |
|
|
| msg103717 - (view) |
Author: Timothee Besset (TTimo) |
Date: 2010-04-20 14:25 |
| My pleasure! We do use a lot of python. |
|
|