| msg57432 - (view) |
Author: Giambattista Bloisi (gbloisi) |
Date: 2007-11-12 23:54 |
| site.py ha two limitations that make difficult to use pth files on my linux installation (gobolinux): - it does not process pth files that are located in directories that are already present in os.path at the time the main method is invoked - it does not process directory recursively Please find attached a patch that solves both. Basically known_paths became a set representing the directories that have been processed. Duplicates in os.path are avoided by looking directly into it. |
|
|
| msg57441 - (view) |
Author: Brett Cannon (brett.cannon) *  |
Date: 2007-11-13 08:11 |
| While I am not sure how I feel about searching every entry on sys.path, I know I don't like the idea of recursing the directory tree. pth files are meant to only be at the top-level of a directory that packages and modules are checked for. |
|
|
| msg57442 - (view) |
Author: Giambattista Bloisi (gbloisi) |
Date: 2007-11-13 08:33 |
| What I meant for recursively was not to scan the full directory tree, but just to scan entries that have been added to the os.path. I think that this was the original intention as http://docs.python.org/inst/search-path.html states: "Any directories added to the search path will be scanned in turn for .pth files". Also http://docs.python.org/lib/module-site.html states: "For each of the distinct head-tail combinations, it sees if it refers to an existing directory, and if so, adds it to sys.path and also inspects the newly added path for configuration files." |
|
|
| msg57468 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2007-11-13 19:00 |
| For what you meant by recursion, that makes more sense. But as for whether this is correct or not, read the next paragraph in the site docs (http://docs.python.org/dev/library/site.html#module-site). It says "A path configuration file is a file whose name has the form package.pth and exists in one of the four directories mentioned above". So the "newly added path" is the four paths mentioned in that paragraph (e.g., the ones that involve sys.prefix and sys.exec_prefix. So the docs do not suggest that any recursive check is done, nor does it need to worry about pre-existing directories. It just needs to check site-packages locations and site-python. If site.py is not checking those directories, it is a bug, otherwise I think the current behaviour matches the documentation. |
|
|
| msg57472 - (view) |
Author: Giambattista Bloisi (gbloisi) |
Date: 2007-11-13 20:47 |
| This make sense. I hope I'm not annoying you. But what I did was to read (http://docs.python.org/inst/search-path.html) before reading (http://docs.python.org/dev/library/site.html#module-site) as the latter is referenced by the first via a html link. The part of my interest of the first document says: "The expected convention for locally installed packages is to put them in the .../site-packages/ directory, but you may want to install Python modules into some arbitrary directory. For example, your site may have a convention of keeping all software related to the web server under /www. Add-on Python modules might then belong in /www/python, and in order to import them, this directory must be added to sys.path. There are several different ways to add the directory. The most convenient way is to add a path configuration file to a directory that's already on Python's path, usually to the .../site-packages/ directory. Path configuration files have an extension of .pth, and each line must contain a single path that will be appended to sys.path. (Because the new paths are appended to sys.path, modules in the added directories will not override standard modules. This means you can't use this mechanism for installing fixed versions of standard modules.)" So the gobolinux maintainer did: it added a file into this dir and it is actually parsed and some dirs are appended to sys.path. And after that the document says what already quoted: "Paths can be absolute or relative, in which case they're relative to the directory containing the .pth file. Any directories added to the search path will be scanned in turn for .pth files. See site module documentation for more information." So why repeat here that some other directories will be scanned for pth files, while you describe the accepted format for paths in .pth files? If the only scanned directories are the ones already cited (a directory that's already on Python's path), IMO there is no need to specify further, unless you mean that the "single path appended to sys.path" that you are describing "will be scanned *in turn* for .pth file". On the other hand the second document doesn't say explicitly that the *only* scanned directories are the ones derived from sys.prefix and sys.exec_prefix. Indeed on darwin (Mac Os X) additional directories are actually parsed. |
|
|
| msg57473 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2007-11-13 22:57 |
| I have made this a documentation bug as obviously there is some needed clarification in how .pth files are handled. If you have any suggested wording, Giambattista, then please feel free to submit patches against the 2.6 docs (easier to work with since they are in reStructuredText format). Otherwise I will try to get to this when I can (which might be a while =). |
|
|
| msg57573 - (view) |
Author: Giambattista Bloisi (gbloisi) |
Date: 2007-11-15 23:16 |
| Please find attached a patch for http://docs.python.org/dev/install/index.html. I think removing a statement is enough to make things clear. About darwin additional directories I'm unsure whether it need further documentation as it seems very platform-related. This is the comment I found in site.py # for framework builds *only* we add the standard Apple # locations. Currently only per-user, but /Library and # /Network/Library could be added too Eventually I used this command in pth file to extend the scanned directories: "import site; site.addsitedir('/System/Links/Libraries/python2.5/site-packages/', set())" I think this is more maintainable and clear than changing site.py, but I don't think it will be forward compatible. |
|
|
| msg57597 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2007-11-17 07:08 |
| Fixed in r59033 for Python 2.6. Thanks for the help! |
|
|