| msg90738 - (view) |
Author: R. David Murray (r.david.murray) *  |
Date: 2009-07-20 18:45 |
| In debugging an intermittent test failure we discovered that if '.' is in the path and import_module is called, then what happens when __import__ causes the creation of a .pyc file changes. Without the forgoing, the permissions of the .py file are copied to the .pyc file (although the fix for issue 6070 removes the execute bits if set). With the forgoing, the write bit is set on the .pyc file even if it was not set on the .py file. The second behavior is how things always worked before 2.6. (Why it changed in 2.6 I don't know, and I'm not convinced it was a change for the better; see also issue 6074). I will upload a test as soon as I create the issue and get an issue number to label it with. Note that this issue exists only 3.x, even though trunk uses the backported import_module function from importlib. |
|
|
| msg90739 - (view) |
Author: R. David Murray (r.david.murray) * |
Date: 2009-07-20 18:47 |
| Test case. |
|
|
| msg90744 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2009-07-20 23:05 |
| import_module are implemented completely differently between 2.6 and 3.1, so the difference is not surprising. As for the permission stuff, the files are simply created using open(..., 'wb') so it has everything to do with what the system default for the executing user happens to be. What exactly are the permissions you are trying to set all bytecode files to? |
|
|
| msg90745 - (view) |
Author: R. David Murray (r.david.murray) * |
Date: 2009-07-20 23:15 |
| If you have a .py file with permissions r--r--r-- and you import it, the pyc file will (in 2.6+ and 3.x) have permissions r--r--r--. However, if '.' is in the path and import_module has been used, the file will instead have permissions rw--r--r-- (assuming the umask is 022). Does that make the issue clearer? As indicated, this is a low priority bug :) |
|
|
| msg90746 - (view) |
Author: R. David Murray (r.david.murray) * |
Date: 2009-07-20 23:25 |
| Oh, a bit of clarification: the call that creates the pyc file in both the "normal" case and the error case is a call to the normal import command (or __import__ in the test case). The call to import_module is a prereq to producing the bug, but it doesn't matter what module it imports (as long as it hasn't been previously imported). The import that shows the behavior imports a TESTFN module in the test case. You might want to load up the test case and play with it. I'm completely mystified as to how import_module could be affecting the regular import semantics...I'm guessing it is a subtle side effect of something unexpected ;) |
|
|
| msg90747 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2009-07-20 23:28 |
| On Mon, Jul 20, 2009 at 16:25, R. David Murray <report@bugs.python.org>wrote: > > R. David Murray <rdmurray@bitdance.com> added the comment: > > Oh, a bit of clarification: the call that creates the pyc file in both > the "normal" case and the error case is a call to the normal import > command (or __import__ in the test case). The call to import_module is a > prereq to producing the bug, but it doesn't matter what module it > imports (as long as it hasn't been previously imported). The import > that shows the behavior imports a TESTFN module in the test case. > > You might want to load up the test case and play with it. I'm > completely mystified as to how import_module could be affecting the > regular import semantics...I'm guessing it is a subtle side effect of > something unexpected ;) It's beyond mystifying as both importlib.__import__ and importlib.import_module are thin wrappers that do nothing but splice strings for the same function that does the heavy lifting. But I will see if I ever find time to work on this. |
|
|
| msg91083 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2009-07-30 04:20 |
| So removing the built-in, frozen, and extension importers did not stop the bug from happening. Calling importlib._bootstrap._PyFileFinder directly does not trigger the bug, even when trying with a finder for '.' first. And having sys.path be only '.' for fileinput still triggers the bug. On another day, the next step is to start stripping out stuff in importlib to see what the minimal thing is that triggers the bug. |
|
|
| msg91300 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2009-08-05 04:44 |
| OK, I need a double-check here, David. At this point I have narrowed it down to this code triggering it:: finder = importlib._bootstrap._PyPycFileFinder('.') sys.path_importer_cache['.'] = finder And I am not kidding, that assignment is required. I might be able to narrow the code down further in _PyPycFileFinder, but I wanted to double- check that I have not gone insane and that these two lines do indeed trigger the problem. |
|
|
| msg92095 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2009-08-31 00:37 |
| Turns out the failure is because I have simply been opening bytecode files for writing using _io.FileIO(..., 'w') which just uses the OS's default permissions. Guess that won't cut it anymore. =) So does this mean I am expected to chmod the bytecode file to have matching read and write bits to the source but no execution bits regardless of the source file? And what about writing new bytecode files? I am not about to ignore permissions and simply write over files just because I can as someone could have set them independently of Python to be read-only. Let me know that I am doing the right thing, David. |
|
|
| msg92096 - (view) |
Author: R. David Murray (r.david.murray) * |
Date: 2009-08-31 01:12 |
| Well, the problem with asking me what the right thing to do is that I think the old behavior of just using the default os umask makes the most sense. I think we should figure out who made the original change in behavior and ask them what the right thing is to do and why. I'll see if I can dig up the 'who' tomorrow. |
|
|
| msg105358 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2010-05-09 01:10 |
| Did this go anywhere, David? Since beta2 just went out now is the time to either revert or enshrine the new behavior. |
|
|
| msg175774 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2012-11-17 16:57 |
| At some point this was fixed in importlib. |
|
|