[Python-Dev] zipimport.c broken with implicit namespace packages (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Sun Jan 3 01:32:24 EST 2016

On 3 January 2016 at 15:29, <mike.romberg at comcast.net> wrote:

" " == Brett Cannon <brett at python.org> writes: > I just wanted to quickly say that Guido's observation as to how > a VFS is overkill is right. Imagine implementing a loader using > sqlite and you quickly realize that doing a dull VFS is more > than necessary to implement what import needs to function. I fear I've made a poor choice in calling this abstract class a VFS (I'm terrible with names). I'm not thinking of anything along the lines of a full file system that supports open(), seek(), read() and everything else. That for sure would be overkill and way more complicated than it needs to be. All I'm really thinking about is a simple abstract interface that is used by an importer to actually locate and retrieve the binary objects that will be loaded. For the simple case I think just two methods would/could server a read only "blob/byte database": listdir(path) # returns an iterable container of "files"/"dirs" found # at path get(path) # returns a bytes object for the given path

We already have the latter: https://docs.python.org/3/library/importlib.html#importlib.abc.ResourceLoader.get_data

It's the former that has taken a while to get to, as the 3rd party pkg_resources module (part of setuptools) already provides a pragmatic API that also has the virtue of being compatible with both Python 2 & 3, and there a few subtleties related to the possible use of temporary files that make a robust API design trickier than it first appears to be.

For folks that are interested in that, folks that aren't following import-sig in addition to python-dev may want to take a look at Brett's design for the importlib.resources API: http://nbviewer.jupyter.org/gist/brettcannon/9c4681a77a7fa09c5347

Cheers, Nick.

P.S. If anyone actually does want a full "virtual file system layer" API for non-filesystem storage locations: http://docs.pyfilesystem.org/en/latest/filesystems.html

-- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia

More information about the Python-Dev mailing list