[Python-Dev] Packaging and binary distributions (original) (raw)

Paul Moore p.f.moore at gmail.com
Mon Oct 31 00:54:25 CET 2011

On 30 October 2011 23:17, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:

Paul Moore <p.f.moore gmail.com> writes:

The MSI format is a little more tricky, mainly because it is a more complex format and (as far as I can tell from a brief check) files are stored in the opaque CAB format, so the only way of getting data out is to do a temporary install somewhere. But I see no reason why that isn't achievable. It's not just about getting the data out of the CAB, though - it's also about integration with Add/Remove Programs and the rest of the Windows Installer ecosystem.

Hang on. I'm talking here about repackaging the binary files in the MSI file for use in a pysetup install invocation. As pysetup has no GUI, and doesn't integrate with Add/Remove, there's no issue here. If you want a GUI and Add/Remove integration, just run the MSI. Or am I missing something? We seem to be at cross purposes here, I suspect I'm missing your point.

1. I will write a 3rd party module to take bsistwininst and bdistmsi modules and install them using packaging It would be important to retain the flexibility currently offered by setup.cfg hooks, as I don't believe any out-of-the-box approach will work for the wide range of use cases on Windows (think Powershell scripts, Visio templates and other Microsoft Office integration components).

Why? Again, if this is purely as a means to consume bdist_xxx files, then the only flexibility needed is enough to cater for any variations in data stored in the bdist_xxx format. The wininst format is easy here - it has directories PLATLIB, PURELIB, DATA, SCRIPTS and HEADERS (corresponding to the installation --install-xxx parameters) and that's all. As long as the module is flexible enough to deal with that, it can read anything bdist_wininst can produce.

I'm also not sure if these formats provide all the flexibility required - e.g. they may be fine for extension modules, but how do they handle packaging include files?

Ah, I think I see what you are getting at. If someone uses the new features and flexibility of packaging to create a fancy custom install scheme, how do they bundle up a binary distribution from that? My (current) answer is that I don't know. The packaging module as it stands only offers the legacy bdist_xxx formats, so the answer is "run pysetup run bdist_wininst on it". If that breaks (as it is likely to - wininst format isn't very flexible) then tough, you're out of luck.

I 100% agree that having a "native" packaging means of building binary distributions from source ones, which captures all of the necessary information to cover any flexibility available to setup.cfg, would be good. But that's potentially a much bigger project than I can manage. My bdist_simple format was based off bdist_dumb/bdist_wininst and had the same limitations as that.

You might be able to get somewhere by running build, then zipping up the whole directory, source, build subdirectory and all. Then on the target machine, unzip and do a --skip-build install. That's a bit of a hack, but should in theory work. Whether it's the basis of a sensible distribution format I don't know.

For 3.3 at least, I'd expect developers to continue distributing bdistwininst or bdistmsi format files. We'll see what happens with binary eggs.

Unix/MacOS users who care will need to propose something themselves. I'm not sure there's anything especially Windows-specific about the bdistwininst format, apart from the prepended GUI executable. One drawback of any current scheme is that if you're packaging an extension module that runs on say Windows, Linux and Mac OS X, there's no easy way to build or distribute a single archive (for a given version of Python, say) which has all the binary variants you want to include, such that at installation time, only the bits relevant to the target platform are installed. The current packaging functionality does sort of support this, but it entails potentially tedious manual editing of the setup.cfg file to add information about what resources apply to which platform - the kind of tedious editing which would be obviated by the right kind of additional support code.

Again, I agree that this would be useful. Not something I have the time to look at though (although if someone else picks it up, I'd be interested in doing some testing and maybe contributing to the work).

I think I now see why we're not understanding each other. I'm coming from the position that the projects I care about (as an end user) use bdist_wininst or bdist_msi at the moment, so all I want is a way of using, as a consumer, those existing distributions (or something equivalent in power) to install the packages via pysetup (which gets me the ability to install in development builds and venvs). I see why a more powerful binary format would be nice for developers, but as an end user I have no direct need for it.

Thanks for your patience. Paul.

More information about the Python-Dev mailing list