[Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ (original) (raw)

Michael Foord fuzzyman at voidspace.org.uk
Wed Nov 3 00:34:38 CET 2010

On 02/11/2010 23:00, Brett Cannon wrote:

On Tue, Nov 2, 2010 at 15:47, Raymond Hettinger <raymond.hettinger at gmail.com> wrote:

On Nov 1, 2010, at 7:35 PM, Brett Cannon wrote:

I think the issue here is that the file structure of the code no longer matches the public API documented by unittest. Personally I, like most people it seems, prefer source files to be structured in a way to match the public API. In the case of unittest Michael didn't. He did ask python-dev if it was okay to do what he did, we all kept quiet, and now we have realized that most of us prefer to have files that mirror the API; lesson learned. But Python 2.7 shipped with this file layout so we have to stick with it lest we break any imports out there that use the package-like file structure Michael went with (which we could actually document and use if we wanted now that Michael has already broken things up). Reversing the trend by sticking all the code into unittest/init.py and then sticking import shims into the existing modules would be a stupid waste of time, especially considering the head maintainer of the package likes it the way it is. I'm not sure I follow where we're stuck with the current package. AFAICT, the module is still used with "import unittest". Yes, as far as you can tell, but who the hell knows what someone is doing with code you are not aware of. As I said, Python 2.7 shipped with the code structured like this, so it's possible someone is importing unittest.case.TestCase instead of unittest.TestCase.

It is also shipped in unittest (and unittest2py3k I might add) so that users of earlier versions of Python can use the new features seamlessly. (unittest2 will be in Django 1.3.)

Much better times to discuss this would be when it was proposed or when it was done, not months after it has been shipped in a production release.

[snip...]

This is in contrast to the logging module which does have a clean separation of components and where it isn't unusual to import just part of the package. What is it you're seeing as a risk that I'm not seeing? Broken imports between Python 2.7 code and any version of Python where unittest is re-merged back into a single module.

I know that some people are using the new package structure directly, because the topic has come up on the Testing in Python mailing list.

Are we permanently locked into the exact ten filenames that are currently used: utils, suite, loader, case, result, main, signals, etc? Is the file structure now frozen? Somewhat, yes. Screwing with unittest is always touchy as absolutely no one wants their tests to break, and that includes messing with imports. We could stick in import shims to shift everything into unittest/init.py, but the benefits you have outlined already don't strike me as not worth the hassle especially since you won't ever get your unittest.py format back. Absolutely, that would be the worst of all possible worlds - a monolithic huge module but still embedded in a package.

People are using the existing package structure to import directly from (against my advice!) as this particular topic has been discussed on the Testing In Python mailing list.

Although Raymond has been vociferous in his detestation of this particular split he does not represent a "clear consensus" in favour of merging back. Both Fred Drake and Barrry Warsaw voiced their approval and on the "Clean up unittest API" issue both yourself (Brett) and Antoine have supported keeping the existing structure.

Alexander Belopolsky and Martin Loewis expressed difficulties with the new structure, but that was in response to the original email from Raymond that didn't seem (on my reading) to expressly suggest re-merging unittest back into a module but was instead seemed to be using it as an example.

I am aware of the costs of having a package rather than single (however large it may be) module, but to my mind the benefits to maintenance still outweigh these cost. I'm happy to again discuss these benefits at great length, but having had the same conversation in person with Raymond twice and at repeated most of the points (but by no means all) in this thread on the mailing list it really feels like going round in circles.

As the maintainer of unittest I'd like to say that in the absence of clear consensus that the merger should happen, or a fiat from the BDFL, the merger won't happen. I believe that this is standard Python development process.

All the best,

Michael

Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk

--

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.

More information about the Python-Dev mailing list