| msg41771 - (view) |
Author: Thomas Heller (theller) *  |
Date: 2002-11-25 18:52 |
| This is not (yet) a complete patch, it is thought as a workspace for improvements to tools/freeze/moduulefinder, so that it can eventually go into the standard library. |
|
|
| msg41772 - (view) |
Author: Thomas Heller (theller) * |
Date: 2002-11-25 18:58 |
| Logged In: YES user_id=11105 First question for anyone listening: isn't the windows _try_registry stuff unneeded, because imp.find_module handles this itself? Even in python15, imp.find_module ("pythoncom") returns 'c:\\winnt\\system32\\pythoncom15.dll' on my system. |
|
|
| msg41773 - (view) |
Author: Thomas Heller (theller) * |
Date: 2002-11-25 19:59 |
| Logged In: YES user_id=11105 Assign to Mark, hoping for an answer. |
|
|
| msg41774 - (view) |
Author: Thomas Heller (theller) * |
Date: 2002-11-26 07:57 |
| Logged In: YES user_id=11105 Mark writes in private email: "Modules listed in the registry was a dumb idea. This whole scheme can die. AFAIK, no one in the world uses it (including win32all since the last build)." |
|
|
| msg41775 - (view) |
Author: Thomas Heller (theller) * |
Date: 2002-11-26 08:11 |
| Logged In: YES user_id=11105 I've removed in current CVS the support for modules listed in the registry. |
|
|
| msg41776 - (view) |
Author: Just van Rossum (jvr) *  |
Date: 2002-11-27 01:26 |
| Logged In: YES user_id=92689 Here's a patch. Summary: any_missing() returns less bogus modules. - I've rewritten scan_code() more or less from scratch, factored bits and pieces out for readability. - keep track of global assignments and failed imports per module; use this to determine whether the Y in "from X import Y" is a submodule or just a global name. This is not 100% doable: you can't tell which symbols are imported when doing a star import of a non-Python module short of actually importing it. - added a new method to ModuleFinder: any_missing_maybe(), which returns *two* lists, one with certain misses, one with possible misses. The possible misses are *very* often false alarms, so it's useful to keep this list separate. any_misses() now simply returns the union of any_missing_maybe(). I pretty sure this is all 100% backward compatible, so after some more testing it could be checked in. |
|
|
| msg41777 - (view) |
Author: Just van Rossum (jvr) * |
Date: 2002-11-27 01:30 |
| Logged In: YES user_id=92689 Oh, I just realized ModuleFinder.report() should still be patched to use any_missing(), or even better, any_missing_maybe(). Will do if this gets through. |
|
|
| msg41778 - (view) |
Author: Thomas Heller (theller) * |
Date: 2002-11-27 08:27 |
| Logged In: YES user_id=11105 Running the current CVS ModuleFinder on a simple script which contains 'import os' with Python 2.2 on Win2k finds 27 modules, while with this patch it only finds 13. (and unassign Mark) |
|
|
| msg41779 - (view) |
Author: Just van Rossum (jvr) * |
Date: 2002-11-27 11:02 |
| Logged In: YES user_id=92689 Whoops, that was a useless patch :-( I managed to forgot to copy the last three lines of the original scan_code() which takes care of delayed imports... Fixed in the new patch. I now also patched Modulefinder.report() to actually use the new any_missing_maybe() feature. (Note for the lib doco to be written: we should document that people should use any_missing[_maybe]() instead of looking directly at mf.badmodules. mf.badmodules is the raw list, any_missing_maybe() filters out the false positives.) I've ran before/after tests on all of Lib/*.py and Lib/test_*.py and the only differences I found are those that this patch address, ie. less false positives when reporting missing modules. |
|
|
| msg41780 - (view) |
Author: Thomas Heller (theller) * |
Date: 2002-11-28 12:04 |
| Logged In: YES user_id=11105 Now that it works really fine, I would like to make progress. This new patch 'thomas1.patch' is against the patched modulefinder, it is easier to read this way. First it passes all changes to self.badmodules through the _add_badmodule() method. Then, and this part is not really polished, but you may get the idea, it introduces a new ignored instance variable which is thought to suppress most of the remaining missing modules warnings. |
|
|
| msg41781 - (view) |
Author: Just van Rossum (jvr) * |
Date: 2002-11-28 18:22 |
| Logged In: YES user_id=92689 What's the difference between this "ignore" variable and the "exclude" list? And why do you need to ignore modules based on importing module? In my OSX appbuilder stuff I simply have a list of modules that will be removed from the 'missing' list when reporting: MAYMISS_MODULES = ['mac', 'os2', 'nt', 'ntpath', 'dos', 'dospath', 'win32api', 'ce', '_winreg', 'nturl2path', 'sitecustomize', 'org.python.core', 'riscos', 'riscosenviron', 'riscospath' ] (I think I could just add this to the exclude list and get the same effect.) Let's please keep it simple! |
|
|
| msg41782 - (view) |
Author: Just van Rossum (jvr) * |
Date: 2002-11-28 20:17 |
| Logged In: YES user_id=92689 Thomas, your _add_badmodule patch actually broke my code... It took me a while to figure out why, and while at it, I discovered a blatant stupidity in my own patch: somehow I completely overlooked the fact that mf.badmodules already references the modules that did the failed import, so my module.badimports thing can go. It does the exact same thing. Almost... and your patch made that clear: the lines that you patched actually make the badmodules reference lie (and with your patch it also makes mod.badimports lie ;-): if module X imports Y.Z, and Z isn't found, it lists Y as an importer of Y.Z, which isn't true. My any_maybe_missing() code needs to know whether the __init__.py tried to failed import _itself_, because if it did (and failed) it's certain that the submodule is missing. But if it _didn't_, the "missing submodule" should be listed as "maybe missing", as it's most likely just a global in the __init__.py. I'll try to revise my patch tomorrow, it should become simpler. In the meantime I think it might indeed be useful to distinguish between "may be missed" and "must be excluded". Is that what your "ignore" patch tried to do? |
|
|
| msg41783 - (view) |
Author: Thomas Heller (theller) * |
Date: 2002-11-28 20:28 |
| Logged In: YES user_id=11105 Just, I trust your analysis while I don't understand it ;-). Thinking more about my "ignore" concept shows that it is unneeded. Warnings can be better removed after modulefinder has been run by examining the data it collects. To answer your question: "excludes" does not mean 'may be missed', it means 'must be excluded' IMO, and no, this was not the purpose of my patch. |
|
|
| msg41784 - (view) |
Author: Just van Rossum (jvr) * |
Date: 2002-11-29 10:16 |
| Logged In: YES user_id=92689 Here's a revised patch. The previous version actually was too lenient: it could list submodules as "maybe missing" when they were missing for sure. Fixed by tracking unresolvable starimports. (Removed the module.badimports attribute of my previous patch, it now uses mf.badimports appropriately.) Meta question: are we going to commit fixes to modulefinder.py in the freeze directory and do the move when the doco is done, or do we not commit anything until we're ready for the move? |
|
|
| msg41785 - (view) |
Author: Neal Norwitz (nnorwitz) * |
Date: 2003-04-12 01:36 |
| Logged In: YES user_id=33168 What's the status of this patch now that modulefinder.py has been moved into the library? |
|
|
| msg41786 - (view) |
Author: Thomas Heller (theller) * |
Date: 2003-04-14 16:53 |
| Logged In: YES user_id=11105 I think it can be closed (although the docs are still missing). |
|
|