[Python-Dev] Dropping init.py requirement for subpackages (original) (raw)

Phillip J. Eby pje at telecommunity.com
Wed Apr 26 21:50:15 CEST 2006

At 11:50 AM 4/26/2006 -0700, Guido van Rossum wrote:

I'm not sure what you mean by "one directory read". You'd have to list the entire directory, which may require reading more than one block if the directory is large.

You have to do this to find an init.py too, don't you? Technically, there's going to be a search for a .pyc or .pyo first, anyway. I'm just saying you can stop as soon as you hit an extension that's in imp.get_suffixes().

> Are you sure you wouldn't rather just write a GoogleImporter class to fix > this problem?

No, because that would require more setup code with a requirement to properly enable it, etc., etc., more failure modes, etc., etc.

I don't understand. I thought you said you had only one top-level package. Fix that package, by putting the code in its init.py. Job done.

> Append it to sys.pathhooks, clear sys.pathimportercache, > and you're all set. For that matter, if you have only one top-level > package, put the class and the installation code in that top-level > init, and you're set to go.

I wish it were that easy.

Well, if there's more than one top-level package, put the code in a module called google_imports and "import google_import" in each top-level package's init.py.

I'm not sure I understand why a solution that works with released versions of Python that allows you to do exactly what you want, is inferior to a hypothetical solution for an unreleased version of Python that forces everybody else to update their tools.

Unless of course the problem you're trying to solve is a political one rather than a technical one, that is. Or perhaps it wasn't clear from my explanation that my proposal will work the way you need it to, or I misunderstand what you're trying to do.

Anyway, I'm not opposed to the idea of supporting this in future Pythons, but I definitely think it falls under the "but sometimes never is better than RIGHT now" rule where 2.5 is concerned. :) In particular, I'm worried that you're shrugging off the extent of the collateral damage here, and I'd be happiest if we waited until 3.0 before changing this particular rule -- and if we changed it in favor of namespace packages, which will more closely match naive user expectations.

However, the "fix the tools" argument is weak, IMO. Zipfile imports have been a fairly half-assed feature for 2.3 and 2.4 because nobody took the time to make the rest of the stdlib work with zip imports. It's not a good idea to change machinery like this without considering at least what's going to have to be fixed in the stdlib. At a minimum, pydoc and distutils have embedded assumptions regarding init modules, and I wouldn't be surprised if ihooks, imputil, and others do as well. If we can't keep the stdlib up to date with changes in the language, how can we expect anybody else to keep their code up to date?

Finally, as others have pointed out, requiring init at the top level just means that this isn't going to help anybody but Google. ISTM that in most cases outside Google, Python newbies are more likely to be creating top-level packages first, so the implicit init doesn't help them.

So, to summarize:

  1. It only really helps Google
  2. It inconveniences others who have to update their tools in order to support people who end up using it (even if by accident)
  3. It's not a small change, unless you leave the rest of the stdlib unreviewed for impacts
  4. It could be fixed at Google by adding a very small amount of code to the top of your init.py files (although apparently this is prevented for mysterious reasons that can't be shared)

What's not to like? ;)

More information about the Python-Dev mailing list