| msg81666 - (view) |
Author: Joshua Bronson (jab) * |
Date: 2009-02-11 20:36 |
| os.makedirs' mode argument defaults to a hardcoded value of 511 (0777 in octal). This forces the caller to either pass in a different hardcoded value (commonly 0750), or to implement a workaround that calculates the expected mode based on the process owner's umask: umask, _ = subprocess.Popen(['sh', '-c', 'umask'], stdout=subprocess.PIPE).communicate() umask = int(umask, 8) mode = 0777 ^ umask os.makedirs(dir, mode) Preferred behavior would be to have the mode default to the value which takes the umask into account rather than the hardcoded value 0777, so that directories would be created with the same permissions as e.g. files created via open(..). N.B. I'm guessing the above workaround won't work on Windows (please excuse my poor Windows knowledge). All the more reason to have os.makedirs calculate the mode properly if none is given. |
|
|
| msg81669 - (view) |
Author: Martin v. Löwis (loewis) *  |
Date: 2009-02-11 20:50 |
| I don't understand the problem. The directory creation *does* take the umask into account - that's the whole point of the umask! The directory being created has the bits (mode & ~umask & 0777) set - because of the semantics of mkdir(2), not because of Python's doing. Closing the report as invalid. |
|
|
| msg81674 - (view) |
Author: Joshua Bronson (jab) * |
Date: 2009-02-11 21:26 |
| Ah, I was misunderstanding the behavior of mkdir, thank you for the explanation. My misunderstanding stemmed from recently coming across two widely-used packages which both pass mode=0750 to os.makedirs. I have no idea why they would be doing this (as it effectively throws away part of the umask), unless they too are misunderstanding the mode parameter. My suspicion is that the mode parameter is widely misunderstood to mean the desired permissions of the created directory. I have filed tickets in the packages in which I came across this to make sure they're doing what they intend. Could the __doc__ for os.mkdir and os.makedirs be expanded to make this clearer? |
|
|
| msg81677 - (view) |
Author: Martin v. Löwis (loewis) * |
Date: 2009-02-11 21:47 |
| > My misunderstanding stemmed from recently coming across two widely-used > packages which both pass mode=0750 to os.makedirs. I have no idea why > they would be doing this (as it effectively throws away part of the > umask), unless they too are misunderstanding the mode parameter. You use such a mode if you want the resulting directory definitely not world-readable, and definitely not group-writable, despite what umask the user running the program may have set. A common case is installation programs, where the installer wants to create directories independent of the umask root may have set when running the installer. > My > suspicion is that the mode parameter is widely misunderstood to mean the > desired permissions of the created directory. I have filed tickets in > the packages in which I came across this to make sure they're doing what > they intend. My suspicion is that people setting explicit file permissions typically know what they are doing, and that you will find that your tickets get closed as invalid, explaining to you that this mode usage is fully intentional. > Could the __doc__ for os.mkdir and os.makedirs be expanded to make this > clearer? I'm skeptical that this should be necessary. Whoever explained umask to you should have mentioned how it really works (i.e. that it is built into and considered by the operating system every time a file or directory is created). By the same pattern, the doc string of mkdir should also explain what relative file names are and how the current directory affects their interpretation. IMO, the doc string shouldn't teach fundamental system principles. |
|
|
| msg81680 - (view) |
Author: Joshua Bronson (jab) * |
Date: 2009-02-11 22:26 |
| > My suspicion is that people setting explicit file permissions > typically know what they are doing, and that you will find that > your tickets get closed as invalid, explaining to you that this > mode usage is fully intentional. For what it's worth, I asked Mike Bayer (author of Beaker and Mako, the packages in question) what the intention was, and the 0750 seems to have been an error: Apparently HTML::Mason (what Myghty was based on) may have originally used 0750 but has since changed it to 0775 (search for "0775" in http://svn.urth.org/filedetails.php? repname=Mason&path=%2Ftrunk%2Flib%2FHTML%2FMason%2FInterp.pm&rev=0&sc=0) . The 0750 propagated from Myghty into Mako and Beaker, but has no apparent reason to be there. For what else it's worth, I've run this by a few other programmers I consider quite smart and experienced (including Ian Bicking), and none of them realized the meaning of the mode parameter either. (If they had I never would have filed this ticket, as I checked with them first!) I agree that knowledge of relative paths should be assumed, but I'm convinced that the mode behavior is widely misunderstood enough to warrant at least one-line "gotcha" in the docstring. </$.02> |
|
|
| msg107764 - (view) |
Author: Sindre Myren (smyrman) |
Date: 2010-06-13 22:59 |
| I have to agree with jab. While the way relative paths works are more or less common knowledge, the way umask does and does not work in relation to python isn't. It is however given a short description at http://docs.python.org/library/os.html#os.makedirs: ...The default mode is 0777 (octal). On some systems, mode is ignored. Where it is used, the current umask value is first masked out. And as it is documented (briefly) online that the umask *is* used, perhaps it is not necessary to add it to the doc-string? |
|
|
| msg107773 - (view) |
Author: R. David Murray (r.david.murray) * |
Date: 2010-06-14 05:44 |
| I don't understand what you mean when you say "how umask works in relation to Python". How it works in relation to Python isn't different from how it works for any other Unix program. Consider, for example, the unix man page for 'open'. I don't think a docstring change is called for; the docstrings are brief summaries to act as memory aids, the real docs are the docs. If someone does want to propose such a change they should open a new issue. |
|
|
| msg107840 - (view) |
Author: Sindre Myren (smyrman) |
Date: 2010-06-14 23:13 |
| >I don't understand what you mean when you say "how umask works in >relation to Python". How it works in relation to Python isn't different >from how it works for any other Unix program. Consider, for example, >the unix man page for 'open'. This is what I mean. The following gnu commands give (on Archlinux): $ umask 0022 $ mkdir test1 $ mkdir test2 --mode 0777 $ ls -l drwxr-xr-x 2 sindrero users 4096 Jun 15 00:59 test drwxrwxrwx 2 sindrero users 4096 Jun 15 00:59 test2 So we see that new folder created with the --mode parameter to gnu's mkdir does not get the umask masked out. The following code in python gives: >> import os >> os.mkdir('test3') >> os.mkdir('test4') >> exit() $ ls -l drwxr-xr-x 2 sindrero users 4096 Jun 15 01:01 test3 drwxr-xr-x 2 sindrero users 4096 Jun 15 01:01 test4 I (as a programmer) have never seen the specific code for python's mkdir function, And I have no way to know whether I should presume that mkdir in python works the same way as the gnu command or not. Unless it is documented that is. Cheers :) |
|
|
| msg107842 - (view) |
Author: Sindre Myren (smyrman) |
Date: 2010-06-14 23:15 |
| *Sorry.. A bit quick there. The line >> os.mkdir('test4') should have been: >> os.mkdir('test4', 0777) |
|
|
| msg107843 - (view) |
Author: Martin v. Löwis (loewis) * |
Date: 2010-06-14 23:20 |
| > I (as a programmer) have never seen the specific code for python's > mkdir function, And I have no way to know whether I should presume > that mkdir in python works the same way as the gnu command or not. > Unless it is documented that is. You should assume that Python's mkdir does the same as Linux' mkdir system call (which is different from the mkdir utility). You can see mkdir's document with "man 2 mkdir", or on http://linux.die.net/man/2/mkdir |
|
|
| msg107847 - (view) |
Author: Sindre Myren (smyrman) |
Date: 2010-06-15 06:49 |
| Thank you very much for the clarification :) |
|
|