Issue 1054967: bdist_deb - Debian packager (original) (raw)
Created on 2004-10-27 00:48 by dairiki, last changed 2022-04-11 14:56 by admin. This issue is now closed.
Messages (43)
Author: Geoffrey T. Dairiki (dairiki)
Date: 2004-10-27 00:48
Here's a first crack at a bdist_deb.
This patch implements two new distutils commands:
bdist_deb: Build Debian packages
debianize: Create and populate a top-level debian subdirectory. (Essentially dh_make for distutils packages.)
There is a slightly detailed README.bdist_deb included in the patches.
I'm open to suggestions for improvements and bug-fixes.
Author: Sean Reifschneider (jafo) * 
Date: 2004-10-27 03:44
Logged In: YES user_id=81797
I'm just doing a review of this code. A couple of things:
There's been some concern expressed about get_default_maintainer. Namely, that if debchange changes it's algorithm, it won't be reflected in this code. It seems like one possible way around that would be to build a directory with a "debian" directory under it, a fake "changelog", and then call debchange to write the data out, and parse it. Too bad there's not a direct hook into debchange to get that information.
Can _formatdate, if email doesn't exist, use rfc822.formatdate()?
Ditto for _parseaddr?
It looks pretty good. However, when trying to build a .deb of my jotweb2 package, it's failing with:
[...] dh_testdir dh_testroot dh_installchangelogs- dh_installdocs cp: cannot stat `doc': No such file or directory dh_installdocs: command returned error code 256 [...]
I'm not sure exactly why. I do have a "doc" directory in my main package directory, but I don't reference to it in my setup.py or MANIFEST. Adding it to the MANIFEST doesn't seem to help this.
Sean
Author: Davide Alberani (alberanid)
Date: 2004-10-27 14:47
Logged In: YES user_id=170840
I've a woody with some packages backported from sarge (debhelper 4.1.90 and dpkg-dev 1.10).
Running python2.3 ./setup.py bdist_deb with some of my projects, I got the error:
dpkg-buildpackage: unknown option or argument --check-dirname-level=1 Debian dpkg-buildpackage .
Commenting out the "check-dirname-level" and "check-dirname-regex" options in the bdist_deb.py file the script can go on, but it exits with the error:
debian/rules:11: *** first argument to `word' function must be greater than 0. Stop.
Hope this helps.
Author: Geoffrey T. Dairiki (dairiki)
Date: 2004-10-29 19:38
Logged In: YES user_id=45814
Thanks for the comments. Here's a second attempt.
Changes include:
It might work with woody. (I'd appreciate it if you could try again, Davide)
'debianize' command renamed to 'dh_make'.
Use debchange to create debian/changelog. This eliminates the need to duplicate debchange's logic to deduce the packagers name and e-mail.
A more complete test script.
Patches are on today's CVS.
Author: Davide Alberani (alberanid)
Date: 2004-10-30 08:57
Logged In: YES user_id=170840
I've tried the 29/10 patch with my home system (dpkg-dev 1.10, debhelper 4.1.90 and dh-make 0.30) and another "pure woody" system (dpkg-dev 1.9.21, debhelper 4.0.2 and dh-make 0.30).
I've used your test suite and some of my personal projects (using both bdist_deb and dh_make); as far as I can tell, it works very well. :-)
bdist_deb always worked as expected, producing good debian packages.
I've used dh_make and then I've run "dpkg-buildpackage -rfakeroot" and I got some errors (but I'm not sure they depends on your code). Also your test suite produced some failures.
Here you can find the output of "dpkg-buildpackage -rfakeroot" and of your tests suite (both were running on my home system): http://erlug.linux.it/~da/tmp/dpkg-buildpackage http://erlug.linux.it/~da/tmp/testsuite
Thank you for your great effort!
Author: Geoffrey T. Dairiki (dairiki)
Date: 2004-11-02 22:20
Logged In: YES user_id=45814
Here's try number three!
Changes are mostly fixes for woody. The README.bdist_deb (included in the patch set) enumerates the changes in more detail.
(I've bitten the bullet and installed woody on one of my machines, so I'm fairly confident it should work now.)
The patches, on todays CVS are, as always, attached below.
==== Note to Davide,
Thank you, again, very much for all the testing.
A couple of the lintian warnings ('prerm-does-not-remove-usr-doc-link' and 'postinst-does-not-set-usr-doc-link') that are listed in your test suite output are due, I think, to your mixed system. Specifically, I suspect you've got the woody version of lintian (which wants to see symlinks from /usr/doc/ to /usr/share/doc/), but a later version of debhelper. (Woody's dh_installdocs (in the debhelper package) automatically generates those links, later versions don't; /usr/doc has been phased out.)
Anyhow, I'm not going to fix those warnings (unless it turns out that you're seeing them on your pure woody system.)
On package with python scripts (in /usr/bin) Woody's lintian generates an 'unusual-interpreter' warning which I think is spurious (i.e. it is not a valid warning.) (Lintian 1.20.17 doesn't recognize things like /usr/bin/python2.1 as a valid interpreter --- it seems valid enough to me.)
Best Regards, Jeff
Author: Geoffrey T. Dairiki (dairiki)
Date: 2004-11-08 21:38
Logged In: YES user_id=45814
Another updated patch set:
As suggested by Bastian Kleineidam, when the source distribution includes it's own debian subdirectory, bdist_deb (rather than just crapping out) will now skip the dh_make stage, and just run debuild.
As always, patches on the current CVS version of distutils are attached below.
Author: Davide Alberani (alberanid)
Date: 2004-11-09 15:39
Logged In: YES user_id=170840
I've created a patch to make your bdist_deb/dh_make commands compatible with Python 1.5.2; it can be found here:
http://erlug.linux.it/~da/tmp/distutils-cvs_debpy152-20041109.patch
Beware that I've tested it for a very short time, so double check everything (especially the permissions on file/umasks).
Moreover, the patch changes also some other files like build_py.py, install_lib.py, bdist_rpm.py, that were not 1.5.2 compatible - these changes are part of another patch I've posted today.
HTH.
Author: Geoffrey T. Dairiki (dairiki)
Date: 2004-11-11 16:18
Logged In: YES user_id=45814
Another updated patch set.
The only change is a bug fix: when using a hand-build set of debian/* files, dh_make will now deduce the correct Debian revision and build architecture. (Bug reported by Bastian Kleineidam.)
Patches, as always are on today's CVS version of distutils.
Author: Marek Kubica (leonidasxiv) *
Date: 2005-01-11 16:53
Logged In: YES user_id=872713
I was looking a long time for something like this. I really hope seeing this in Python 2.5, although I'm not familiar with Python's development process.
Author: Davide Alberani (alberanid)
Date: 2005-02-23 11:10
Logged In: YES user_id=170840
Sometimes you need to install the same package for many different releases of Python; unfortunately this will fail, because dpkg refuses to override the dummy man page, present in both packages. Would it be a good idea to modify line 453 of dh_make to include pyver in the name of the manpage file?
Author: Kurt B. Kaiser (kbk) *
Date: 2005-03-19 23:18
Logged In: YES user_id=149084
In response to jafo's request for comments on python-dev:
This facility should be documented in the Python Module docs. A section should be introduced in "Distributing Python Modules" after the 5.2 Creating RPM Packages. Also, the reference disutils.command.bdist_deb should be supplied. The fact that the rest of distutils is 'lightly' documented is no excuse :-)
The README.bdist_deb should be folded into the doc strings for the two modules (except the change info should be left behind in the patch).
The existing doc strings in the two modules are not always in standard form, and could be more extensive.
There are triple quoted block comments in the code. These should be # comment blocks, cf. PEP8 and PEP 257. You don't use doc strings to document variables.
deb_util.py probably belongs in distutils/command/
I would like it better if the default was to build the source package, and that would be consistent with bdist_rpm's action.
I'm not entirely happy with the overall design: it seems there are two ways to go. First, run bdist_deb, which give you a package which isn't debian compliant; there's no debian/changelog. Second, you can run dh_make, which will 'debianize' the Python source. This can only be done once, and then the recommendation is to use the standard Debian tools to create the packages. In this second case, apparently you can't use bdist_deb, nor do you have a convenient way of updating when the Python code changes, if I understand the README.bdist_deb correctly. If these impressions are correct, it's a lot of code for relatively little effect, at least at its current level of development.
What about alberanid's last comment?
Do we have a commitment from Dairiki to maintain this code for at least a couple of years?
Author: Martin v. Löwis (loewis) *
Date: 2005-03-28 15:06
Logged In: YES user_id=21627
I agree with most of kbk's comments, except for 5: all of the *_util modules are in distutils, why not deb_util? It is not a command.
Also, dairiki, are you willing to sign the contributor form? see
http://www.python.org/psf/contrib.html
Author: Andrew Straw (astraw)
Date: 2007-04-03 03:17
You may also be interested in stdeb, which produces Debian source packages from Python packages. See http://stdeb.python-hosting.com/
Author: Sean Reifschneider (jafo) *
Date: 2007-06-10 10:58
This code has a number of outstanding issues including (primarily) whether a contributors agreement will be signed. It would be nice to have this code in Distutils, but I think this is a show-stopper to getting it in. If Geoffrey signs a contributor agreement, I'll open it back up and see if I can address the other outstanding issues.
Sean
Author: Davide Alberani (alberanid)
Date: 2007-06-10 11:11
jafo wrote:
It would be nice to have this code in Distutils
Absolutely! I think that a deb packages generator (so policy-compliant as it can be) will be useful to many people.
Unfortunately I'm not an expert about debian packages, but maybe I can do some testing and/or write some code.
Author: Tarek Ziadé (tarek) *
Date: 2009-02-17 22:36
Sean,
I am re-opening this issue, to study the inclusion of a bdist_deb command for 2.7/3.1, since there's some interest in the community.
I have taken the assigment since it was rejected, but let me know if you want me to assign back to you of course.
Author: Lukas Lueg (ebfe)
Date: 2009-02-23 11:07
Count me in
Author: Paul Hummer (rockstar)
Date: 2009-05-02 22:56
I've been doing a review of this patch for the last few hours. There are a few issues that need to be taken care of in order for it to move forward, and I'm currently working on them, in this order:
- Update the patch to the most recent svn. The most current patch is almost 5 years old. I've ported it forward.
- Fix the unittests. The patch is failing its unittests currently, and so I'm fixing the unittests, and will be adding tests where there may not be any coverage.
- Implement the suggestions made by alberanid and kbk, adding tests where necessary.
I've been chatting with jafo, and am willing to commit to maintaining this code for the next few years, and am willing to sign the contributor agreement in order to do so. I don't particularly want to step on anyone's toes, so if someone else is also working on this, I'm happy to hand off what I've already been working on.
Author: Lukas Lueg (ebfe)
Date: 2009-05-04 12:02
Thanks for your efforts. I don't think you are stepping on anyone's toes when picking up an issue that was unsolved for almost 5 years :-)
Please post patches to this bug for review/comments/help/whatever
Author: Andrew Straw (astraw)
Date: 2009-10-04 20:58
stdeb ( http://github.com/astraw/stdeb ) now includes a bdist_deb distutils command.
Author: Tarek Ziadé (tarek) *
Date: 2009-10-26 22:17
I am closing this issue, since stdeb implements this command now.
Also, I'd love to add it in the distutils documentation, but there's one issue.
Andrew, do you think you could have a plain distutils version of this command ?
Setuptools changes the Command and the Distribution classes, and affects Distutils global behavior, so we can't document its usage in the Distutils documentation as it is.
Author: Andrew Straw (astraw)
Date: 2009-12-31 00:12
As of stdeb 0.5.0, the setuptools dependency has been removed. Now stdeb requires only plain distutils.
Author: Barry A. Warsaw (barry) *
Date: 2010-04-27 20:02
astraw: I've been playing with stdeb. I think it's a nice implementation of bdist_deb, but it doesn't seem to include the dh_make/debianize functionality of this patch. Is that the case or have I missed something?
Author: Éric Araujo (eric.araujo) *
Date: 2010-04-27 20:14
Hello
There has been a number of discussions about bdist_deb, and some code too. I don’t have links handy, unfortunately, but I remember a conclusion that was: Don’t.
Debian packages are best made by Debian tools, which go to great lengths to comply with Debian Policy, work together, and are frequently updated. On the Distutils(2) site, the most we can do would be a bdist_dsc command that produces a special file which is used by debianizing tools.
A key thing to remember is that .deb is not just a file format, it’s the embodiment of a Policy (the real value of Debian is its policy, not its package manager—a nice technical side effect), and as such, best handled by Debian tools.
Disclaimers: I’ve not been following packaging discussions for long, these lines are a summary of other people’s thoughts, I’m not a Debian developer or maintainer.
Cheers
Author: Barry A. Warsaw (barry) *
Date: 2010-04-27 20:22
There are really two aspects of this patch. At the moment, I'm less interested in bdist_deb than I am in the 'debianize' (i.e. dh_make) functionality. I think 'python setup.py debianize' (or whatever) would be a nice way to jump start a debian/ directory from the available metadata.
Author: Éric Araujo (eric.araujo) *
Date: 2010-04-27 20:26
Why not use Debian’s dh_make?
Author: Barry A. Warsaw (barry) *
Date: 2010-04-27 20:38
That's essentially what I want, except I want to feed dh_make some better defaults based on metadata that already lives in setup.py.
Author: Éric Araujo (eric.araujo) *
Date: 2010-04-28 00:24
Hello
Thanks for clarifying your point.
Note that I had quickly tested dh_make in a near-empty directory before posting, and the only question I got asked was the type of the package (library, many libraries, program, etc). But I’ve just realized that I have two dh_make-related variables set in my environment, plus maybe config files, so a typical user may have more questions to answer, which is annoying if the setup.cfg¹ already has them. And to be honest, the question about the type of the package is not simple if you’re not fluent with the Policy, so if Distutils2 can answer that thanks to its knowledge, all the better. I therefore agree with you :)
One thing I’m attached to: The name of the command should respect existing conventions. Entirely new concepts get dedicated names (e.g. “check” and “test”), but commands fitting in the areas of sdist or bdist should say so in their name. How about sdist_debian?
It may be useful to precise something at this point: You may read about both “debian” and “Debian” packages. “debian” or “.deb” or “debs” are binary distributions that use the file formats defined by Debian, whereas “Debian” packages are binary distributions that use these formats, conform to the Debian policy, are tested on lintian.debian.org and piuparts.debian.org, and most importantly, are scrutinized for Debian Free Software Guidelines compliance. That’s why I, for one, never uses third-party deb packages or apt repos: If it’s going to be installed system-wide, I want a verified Debian package.
HTH
¹ Not a typo, just an anticipation of PEP 390 ;)
Kind regards
Author: Barry A. Warsaw (barry) *
Date: 2010-04-28 19:59
I'm not so sure about sdist_debian for the command I'm thinking about because it doesn't actually build a distribution. It just creates a 'debian' directory so I think I like just 'debian' as the name of the command. But thanks for the feedback; time to write some code!
Author: Andrew Straw (astraw)
Date: 2010-04-29 06:40
Barry, stdeb does much of what you're describing. (Try the "python setup.py sdist_dsc" command.)
I'm not particularly pleased with the stdeb codebase as it stands, but it does work reasonably well, and I'd like to see progress in this domain, but I'm afraid I don't understand the difference between what it offers and what you're proposing. I guess maybe you're not aware of the "sdist_dsc" distutils command it offers?
-Andrew
Author: Barry A. Warsaw (barry) *
Date: 2010-04-29 12:12
Hi Andrew,
Yes, I've played with sdist_dsc, and it's nice in that it takes a non-Debianized source tree and turns it into a source package that can be uploaded. What I'm looking for is something that can take a non-Debianized source tree and turn it into a native package, essentially by pre-filling a debian/ directory like dh_make does, but including additional metadata available in the setup.py.
I've started taking a crack at this:
https://launchpad.net/setupdebian
Author: Andrew Straw (astraw)
Date: 2010-04-29 18:21
Barry, I'm sorry I still don't understand what you think is the essential distinction. A debian source package (.dsc, .orig.tar.gz, and .diff.gz files) is simply made from a (patched) source directory, including the debian/ directory, and an upstream tarball. To build this sdist_dsc must of course make the debian/ directory. Thus, it "takes a non-Debianized source tree and turn it into a native [Debian source] package, essentially by pre-filling a debian/ directory like dh_make does, but including additional metadata available in the setup.py". Then, once it done this, it calls "dpkg-buildpackage -S" to turn this filesystem structure into the actual source package. If your final goal is simply the filesystem structure rather than the Debian source package (.dsc, .orig.tar.gz and .diff.gz), then it seems like it would be easier to refactor stdeb to stop before calling dpkg-buildpackage rather than writing a whole new alternate way of generating the debian/ directory based on the contents of setup.py.
So I'm afraid I still don't understand what you want setupdebian to do that stdeb does not.
Author: Barry A. Warsaw (barry) *
Date: 2010-04-29 18:46
Hi Andrew,
I want debian/ to be created in . (aka, cwd, the directory containing setup.py), not in ./deb_dist/package-version
Author: Andrew Straw (astraw)
Date: 2010-04-29 19:12
I see. So is copying the debian/ directory into its desired location a possibility for you, either manually or via a new distutils commands that shares 99% of its code with sdist_dsc? It doesn't seem like enough of a difference to start a new project.
Author: Barry A. Warsaw (barry) *
Date: 2010-04-29 19:23
Manual moving/copying is what I want to avoid. I'd be totally happy with not reinventing the wheel if you think this would be easy to add to stdeb! Specifically: just create debian/ in place and do not build package.
I have a few other minor suggestions, though if you think this isn't the best place to track them, please let me know where to submit them.
- Add setup.py entry_points so you don't have to use --command-packages
- Package: shouldn't change dots to dashes. E.g. flufl.enum should be called python-flufl.enum
- Update to Standards-Version: 3.8.3
I'm happy to test whatever you come up with.
Author: Éric Araujo (eric.araujo) *
Date: 2010-04-29 19:26
tree and turns it into a source package that can be uploaded. What I'm looking for is something that can take a non-Debianized source tree and turn it into a native package, essentially by pre-filling a debian/ directory like dh_make does, but including additional metadata available in the setup.py.
I've started taking a crack at this:
Barry, what do you mean with “native package”? In Debian parlance, a native package is a piece of software that isn’t released outside of Debian and thus has no hyphen in the version number (a hyphen is normally used to separate the upstream version number from a debian-packaging-specific number, e.g. “0.2-1”). I don’t know stdeb (I avoid Setuptools), so I don’t understand the difference you make.
On a related note, I’ve seen in your setupdebian code that you unconditionally remove some files under the debian directory, but some of them may be useful, especially for building multiple binary packages (i.e. separate installable .deb packages) out of one source package, e.g. to split architecture-independent modules from arch-dependent C extensions (like mercurial and mercurial-common). More info:
- how to build multiple binary packages: http://wiki.debian.org/PkgSplit http://lists.debian.org/debian-python/2009/09/msg00091.html
- questioning usefulness of PEP 345 for automation of deb packaging: http://lists.debian.org/debian-python/2009/11/msg00076.html
- extensions to Distutils that may meet some debian packaging needs: http://bazaar.launchpad.net/~python-distutils-extra-hackers/python-distutils-extra/debian/files
- rough proposal by myself for support in Distutils2: http://wiki.python.org/moin/Distutils/Proposals/SplitAndCompoundDistributions
HTH
Author: Barry A. Warsaw (barry) *
Date: 2010-04-29 19:35
Eric, you're right, "native" isn't exactly what I was getting at. I want the debian/ directory created right next to the setup.py, as I've described in some follow ups.
As far as what you see in setupdebian, this is just preliminary experiments using test-driven development. I had intended to support switches for other types of packaging options, but first things first. Since Andrew is willing, I'd rather work with him to get some of the features I'd like to see rather than continue with a separate package, so if that proves fruitful, I'll abandon setupdebian.
BTW, have you seen the python-debian package? It seems to have a lot of useful stuff for manipulating changelogs, control files and such.
Author: Éric Araujo (eric.araujo) *
Date: 2010-04-29 20:16
Barry, IIUC you want the “debian” command to create a debian source package, which can be converted to debian binary package(s), checked and uploaded with standard tools. Perfect.
Andrew, I am uncomfortable with stdeb. (Trying to express respectful constructive critique, not just picking on your project.) The version in Debian testing, 0.5.1-1, still requires setuptools (actually only pkg_resources, which is split into another package in Debian, for space or politic reasons). The py2dsc script seems to be obsoleted by the to-be “debian” command. The pypi-install script seems dangerous to me, because downloading unverified software and putting it into the system seems a rather bad idead (a.k.a. “sudo ./setup.py install considered harmful”).
python-debian provides useful support code for generating and reading debian formats. A quick read show the code would benefit from some 2.6 modernizing (i.e. gain in simplicity and readability by using modern idioms and stdlib functions). Would using this package in stdeb / distutils2.command.debian instead of shelling out bring anything?
(Sorry about the badly wrapped paragraph I forgot to remove in my previous message. And sorry to write so much words instead of code ;)
acute-accent-ly yours
Author: Barry A. Warsaw (barry) *
Date: 2010-04-29 21:19
Éric, sorry about missing that accent. /me needs a bigger font. :)
RE: python-debian modernization. We'd have to check with the authors to see what version of Python they need to remain compatible with. If it's 2.6, then I'm all for modernizing.
I'm not a huge fan of shelling out to get the debian directory, but dh_make seems to be what people want to emulate. I was planning on using python-debian to fill out e.g. debian/control with information that dh_make couldn't easily be convinced to create.
Author: Andrew Straw (astraw)
Date: 2010-05-04 00:29
I have moved the recent discussion on stdeb to a thread on the distutils-sig.
Author: Éric Araujo (eric.araujo) *
Date: 2010-05-07 21:39
We forgot to reopen the bug.
Author: Tarek Ziadé (tarek) *
Date: 2010-05-07 21:53
Please don't. I've closed this bug because the bdist_deb command is currently managed by the stdeb project and it's fine like htis. Distutils2 will not have specialized linux bdist_* command like this one, because it is best to keep it in a separate project that can evolve as its own pace, in accordance with debian/ubuntu cycles.
History
Date
User
Action
Args
2022-04-11 14:56:07
admin
set
github: 41087
2010-10-25 13:06:31
eric.araujo
set
stage: resolved
resolution: rejected
versions: + 3rd party, - Python 2.6, Python 2.5, Python 3.1, Python 2.7, Python 3.2, Python 3.3
2010-05-07 21:53:39
tarek
set
status: open -> closed
messages: +
2010-05-07 21:39:29
eric.araujo
set
status: closed -> open
messages: +
2010-05-04 00:29:44
astraw
set
messages: +
2010-04-30 21:08:52
piotr
set
nosy: + piotr
2010-04-29 21:19:23
barry
set
messages: +
2010-04-29 20:16:27
eric.araujo
set
messages: +
2010-04-29 19:35:25
barry
set
messages: +
2010-04-29 19:26:01
eric.araujo
set
messages: +
2010-04-29 19:23:54
barry
set
messages: +
2010-04-29 19:12:32
astraw
set
messages: +
2010-04-29 18:46:10
barry
set
messages: +
2010-04-29 18:21:45
astraw
set
messages: +
2010-04-29 12:12:39
barry
set
messages: +
2010-04-29 06:40:16
astraw
set
messages: +
2010-04-28 19:59:29
barry
set
messages: +
2010-04-28 00:24:08
eric.araujo
set
messages: +
2010-04-27 20:38:55
barry
set
messages: +
2010-04-27 20:26:50
eric.araujo
set
messages: +
2010-04-27 20:22:19
barry
set
messages: +
2010-04-27 20:14:27
eric.araujo
set
versions: + Python 2.6, Python 2.5, Python 3.2, Python 3.3
nosy: + eric.araujo
messages: +
components: + Distutils2, - Distutils
2010-04-27 20:02:59
barry
set
nosy: + barry
messages: +
2009-12-31 00:12:53
astraw
set
messages: +
2009-10-26 22:17:41
tarek
set
status: open -> closed
messages: +
2009-10-04 20:58:35
astraw
set
messages: +
2009-05-04 12:02:13
ebfe
set
messages: +
2009-05-02 22:56:21
rockstar
set
nosy: + rockstar
messages: +
2009-04-07 11:05:42
anadelonbrin
set
nosy: + anadelonbrin
2009-02-23 11:07:54
ebfe
set
nosy: + ebfe
messages: +
2009-02-17 22:36:07
tarek
set
status: closed -> open
versions: + Python 3.1, Python 2.7, - Python 2.5
nosy: + tarek
messages: +
assignee: jafo -> tarek
resolution: rejected -> (no value)
2004-10-27 00:48:46
dairiki
create