| msg46957 - (view) |
Author: Alyssa Coghlan (ncoghlan) *  |
Date: 2004-09-27 14:22 |
| Implements the '-m' option recently discussed on python-dev. Runs a module 'as if' it had been invoked from the command line. E.g., for a build directory, the following two commands are equivalent: ./python -m pdb ./python Lib/pdb.py Note that neither of these does the same thing as "./python -c "import pdb". (In this latter case, any "if __name__ == "__main__" code would not be executed, whereas it will be invoked in the first two cases). Given the vagaries of sys.path, this is quite handy - if a module can be imported, it can be easily used as a script too. Not that all modules will necessarily do anything _interesting_ when used as a script. . . The current implementation makes changes to main.c (handle the new argument), pythonrun.[ch] (new flavour of PyRun_) and import.[ch] (expose a PyImport_FindModule C API) Current limitations / to-do if this is pursued: - print sensible errors for non-Python files (e.g. hotshot) - allow 'dotted' names - decide what to do with packages (e.g. run __init__.py) - check usage messages in main.c are all still under 512 characters (I assume that limit was set for a reason) - the 1K limit on absolute file paths seems rather arbitrary. - this is code written in the wee hours of the morning, so it probably has a few other lurking 'features'. |
|
|
| msg46958 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2004-09-30 09:39 |
| Logged In: YES user_id=1038590 New version of patch attached (run_module_2.diff). This version allows modules within packages to be executed. It does this *without* loading the packages first (just as if the script were specified directly on the command line). It's also a little more responsive when it comes to error message, and I've confirmed that the usage message components are now under 512 bytes each. The FindModule method is now marked as private (i.e. with leading underscore). The limit on file paths in pythonrun.c is now taken from osdefs.h (the same limit that import.c already uses) The only bug I know of is that it misbehaves if you supply a name like "a.b.module.d". You will get an ImportError saying that d was not found, and the file pointer for 'module' won't get closed properly. |
|
|
| msg46959 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2004-09-30 09:48 |
| Logged In: YES user_id=1038590 Barry, kicking this in your direction solely because you were one of the people to comment on it on py-dev. Feel free to say 'too busy' :) |
|
|
| msg46960 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2004-10-01 02:51 |
| Logged In: YES user_id=1038590 New version (run_module_3.diff) Takes more account of the file description reported by PyImport. Explictly restricts itself to PY_SOURCE and PY_COMPILED scripts. Behaves correctly when a non-package is encountered while parsing a dotted name (closes the file descriptor and reports an error). I also realised that the '-m' option doesn't currently comprehend "__path__" variables in packages. So maybe we *should* be loading the packages as we find them, and then querying them for their __path__ variable. |
|
|
| msg46961 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2004-10-01 04:13 |
| Logged In: YES user_id=1038590 New version (#4) Trivial fix - tell PyRun_SimpleFile to close the file when it's done with it. |
|
|
| msg46962 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2004-10-04 14:30 |
| Logged In: YES user_id=1038590 Relevant documentation locations that I can find: Tutorial section 2.1 (Invoking the Interpreter) Add a very brief description of the '-m' option Python/C API section 2 (The Very High Level Layer) Document PyRun_SimpleModule What's New sections 12 & 13 I'll do this as a separate patch from the previous two, as I can't recall if Andrew prefers to write these himself. |
|
|
| msg46963 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2004-10-04 17:48 |
| Logged In: YES user_id=1038590 5th version of patch attached - restricts option to top-level modules only. The correct semantics for a module inside a package just aren't clear enough at this stage. Support for dotted names can be added later if this patch gets accepted, and it still seems like a good idea. - adds a missing check from main.c (if we got the module option, don't try to open the file). |
|
|
| msg46964 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2004-10-04 18:21 |
| Logged In: YES user_id=1038590 Doc patch attached. Take it with a grain of salt, since this is the first real doc patch I've made. This doesn't cover anything for 'What's New' - that can wait until the patch gets accepted (if it does). |
|
|
| msg46965 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2004-10-04 23:05 |
| Logged In: YES user_id=1038590 Raymond, Passing to you to see what you think of the code, rather than just the feature in the abstract. Guido settled for not vetoing it, but the discussion with him allowed me to articulate why I think this is a worthwhile feature. (Details at http://mail.python.org/pipermail/python-dev/2004-October/049236.html) Extension to allow packages and zip imports can wait until there is any demand for either feature. I don't really expect to see any, since I think the main uses will be "python -m pdb", "python -m profile" and "python -m timeit". |
|
|
| msg46966 - (view) |
Author: Raymond Hettinger (rhettinger) * |
Date: 2004-10-05 04:32 |
| Logged In: YES user_id=80475 Armin, do you have time for this one. I'm buried for a few days. |
|
|
| msg46967 - (view) |
Author: Armin Rigo (arigo) * |
Date: 2004-10-05 13:15 |
| Logged In: YES user_id=4771 Sorry, I didn't follow any of this discussion, neither here nor in python-dev, so I cannot really comment. |
|
|
| msg46968 - (view) |
Author: Raymond Hettinger (rhettinger) * |
Date: 2004-10-06 10:58 |
| Logged In: YES user_id=80475 The patch does not compile on windows. AFAICT, importdl.h is the culprit because it loads windows.h for a second time resulting a double redinition of BYTE deep in the bowls of include files. Wish I had time to help you on this one, but I don't. |
|
|
| msg46969 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2004-10-06 15:07 |
| Logged In: YES user_id=1038590 New version attached which should fix the Windows compile bug (pythonrun.c does the right thing by undefining BYTE before including windows.h - the inclusion of 'importdl.h' has been moved to be after that 'fixed' include). Unfortunately, I won't get access to a Windows build machine until after the beta 1 deadline, or I'd check it myself :( |
|
|
| msg46970 - (view) |
Author: Raymond Hettinger (rhettinger) * |
Date: 2004-10-06 18:09 |
| Logged In: YES user_id=80475 Put the test directory in PYTHONPATH and run the following to see if you get the same results: python -tt -m regrtest -u network python -tt lib\test\regrtest.py -u network |
|
|
| msg46971 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2004-10-06 18:56 |
| Logged In: YES user_id=1038590 Nice catch. . . I figured out what the problem is - regrtest.py uses sys.argv[0] to delete it's own directory from sys.path. And -m currently sets sys.argv[0] to just 'regrtest', instead of the full path to the module. Obviously, that is going to have to change. Either PyRun_SImpleModule will have to do it, or I'll get rid of that function all together, and put the 'module-finding' logic directly in main.c. |
|
|
| msg46972 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2004-10-06 21:32 |
| Logged In: YES user_id=1038590 New version (7) attached which does the right thing for "python -tt -m regrtest -u network". (I really like that test case btw - very concise) This one sets argv[0] correctly, but was initially a little ungraceful in some failure modes (such as "python -m parser"). I had to add a new private API function to import.h (_PyImport_IsScript) to clean that up. Examples of the error handling in this version: [...@localhost src]$ ./python -m foo ./python: module foo not found [...@localhost src]$ ./python -m sys ./python: module sys has no associated file [...@localhost src]$ ./python -m parser ./python: module parser not usable as script (/home/.../dev/python/dist/src/build/lib.linux-i686-2.4/parser.so) |
|
|
| msg46973 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2004-10-06 22:24 |
| Logged In: YES user_id=1038590 New version of doc patch which changes the tutorial only (since PyRun_SimpleModule is no longer part of the patch). |
|
|
| msg46974 - (view) |
Author: Raymond Hettinger (rhettinger) * |
Date: 2004-10-06 23:56 |
| Logged In: YES user_id=80475 Please put in a single, combined documentation patch. Be sure to spell-check and run: python -m texcheck tut.tex |
|
|
| msg46975 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2004-10-07 03:36 |
| Logged In: YES user_id=1038590 texcheck complains about the Unicode in the tutorial even on a clean checkout. The line numbers (> 900) are well past the point where I'm editing. (It turns out editing the the tex files works a lot better with xemacs than with the KDE editors - no spurious edits this time) I'm not sure what, if anything, I need to do for "what's new", but I've uploaded a combined code & documentation patch that implements version 7 of the -m behaviour, and makes relevant changes to section 2 of the tutorial. |
|
|
| msg46976 - (view) |
Author: Raymond Hettinger (rhettinger) * |
Date: 2004-10-07 06:46 |
| Logged In: YES user_id=80475 Accepted and applied. Checking in Doc/tut/tut.tex; /cvsroot/python/python/dist/src/Doc/tut/tut.tex,v <-- tut.tex new revision: 1.254; previous revision: 1.253 done Checking in Include/import.h; /cvsroot/python/python/dist/src/Include/import.h,v <-- import.h new revision: 2.30; previous revision: 2.29 done Checking in Misc/NEWS; /cvsroot/python/python/dist/src/Misc/NEWS,v <-- NEWS new revision: 1.1154; previous revision: 1.1153 done Checking in Modules/main.c; /cvsroot/python/python/dist/src/Modules/main.c,v <-- main.c new revision: 1.84; previous revision: 1.83 done Checking in Python/import.c; /cvsroot/python/python/dist/src/Python/import.c,v <-- import.c new revision: 2.240; previous revision: 2.239 cvs diff: [06:46:25] waiting for rhettinger's lock in /cvsroot/python/python/dis t/src/Doc/tut done |
|
|
| msg46977 - (view) |
Author: Thomas Heller (theller) * |
Date: 2004-10-08 18:23 |
| Logged In: YES user_id=11105 Regarding support for packages, here's a use case: python -m pychecker.checker mymodule.py (plus my own tools which live inside packages) |
|
|
| msg46978 - (view) |
Author: Alyssa Coghlan (ncoghlan) * |
Date: 2004-10-08 22:53 |
| Logged In: YES user_id=1038590 I've opened a new tracker item to cover enhancement to support packages. http://sourceforge.net/support/tracker.php?aid=1043356 |
|
|