This patch adds two improvements to freeze/modulefinder. 1. ModuleFinder now keeps track of which module is imported by whom. 2. ModuleFinder, when instantiated with the new scan_extdeps=1 argument, tries to track dependencies of builtin and extension modules.
Logged In: YES user_id=21627 I dislike the chunk on finding external dependencies. What is the typical use case (i.e. what module has what external dependencies)? It seems easier to hard-code knowledge about external dependencies into ModuleFinder; this hard-coded knowledge should cover all core modules. In addition, there should be an API to extend this knowledge for non-standard modules. Furthermore, by executing an import, you cannot be sure that you really find all dependencies - some may only show up when a certain function is used.
Logged In: YES user_id=11105 The use case is to find as much dependencies as possible. Sure, you cannot assume that importing an extension module finds all dependencies - only those which are executed inside the initmodule function. OTOH, this covers a *lot* of problematic cases, pygame and numpy for example. The situation is (somewhat) similar to finding dependencies of python modules - only those donewith normal import statements are found, __import__, eval, or exec is not handled. A possible solution would be to run the script in 'profiling mode', where the script is actually run, and all imports are monitored. This is however far beyond ModuleFinder's scope. Hardcoding the knowledge about dependencies into ModuleFinder for the core modules would be possible although inelegant IMO. An API for non-standard modules would be possible, but how should this be used without executing any code?
Logged In: YES user_id=21627 I would still prefer the inelegant approach. Your approach appears to be quite dangerous: the packager would essentially run arbitrary C code... If you absolutely have to use such a feature, I think you can do better than analysing the python -v output: watch sys.modules before and after the import. As for extending the hard-coded knowledge: I was suggesting that the packaging tool that uses modulefinder has a mechanism to extend the hard-coded knowledge by other hard-coded knowledge (which lives in the packaging tool, instead of living in modulefinder). If the packaging tool absolutely wants to, it also could run the Python interpreter through a pipe and put the gathered output into modulefinder :-)
