Bootstrapping Debian packages

I read something about dh_make again a while ago, which made me wonder, besides dh_make, how do people generally bootstrap new Debian packages. I don't think I've ever actually used dh_make for anything but experimenting. For the longest time, copying over the packaging files from some similar package worked pretty well.

Recently, however, I have applied a completely new strategy: I start with nothing, call debuild, fix the error, retry, until it works. This might sound pretty bizarre, but it works pretty well and enhances your understanding of how the packaging tools work.

Here is about how it works:

Start with nothing, call debuild, it complains:

cannot find readable debian/changelog anywhere!

Call dch --create, which says:

Cannot find debian directory!

OK, mkdir debian, call dch --create again, fill in the template. Call debuild again, now it says:

but there's no debian/rules there!

Create a debian/rules, that's easy to do by heart nowadays, at least as a start:

#!/usr/bin/make -f

%:
        dh $@

Call debuild again, now dpkg-buildpackage actually starts, but stops with:

dpkg-source: error: cannot read mypackage/debian/control: No such file or directory

At this point I'm too lazy to figure out what is necessary to put into debian/control, so let's just keep it simple and touch debian/control. At this point dpkg-source gets into a bit of trouble:

Use of uninitialized value $sourcepackage in sprintf at /usr/bin/dpkg-source line 290.

So let's put that in and also follow the subsequent advice to add debian/source/format and the Maintainer and Standards-Version fields. So far we have:

Source: mypackage
Maintainer: Peter Eisentraut <petere@debian.org>
Standards-Version: 3.9.2

The next hint of a message is:

dh: No packages to build.

This means we need a binary package stanza, so I'll just add

Package: mypackage
Architecture: any

Now debhelper complains:

dh: Compatibility levels before 5 are deprecated.

Not sure why it didn't complain about that earlier. Let's stick 8 in there.

At this point I had to do actual work and mess around with debian/rules a bit to get the package to actually build, but a few minutes later I had a functioning provisional package.

The next step is to clean up the warnings from the various dpkg-* workers:

dpkg-gencontrol: warning: missing information for output field Description
dpkg-gencontrol: warning: package mypackage: unused substitution variable ${shlibs:Depends}
dpkg-deb: warning: parsing file 'debian/mypackage/DEBIAN/control' near line 6 package 'mypackage': missing description
dpkg-genchanges: warning: missing Section for binary package mypackage; using '-'
dpkg-genchanges: warning: missing Priority for binary package mypackage; using '-'
dpkg-genchanges: warning: missing Section for source files
dpkg-genchanges: warning: missing Priority for source files

So we add Description, Depends, Section, and Priority.

And finally we have a list of complaints from Lintian to address:

W: mypackage source: debhelper-but-no-misc-depends mypackage
E: mypackage source: package-uses-debhelper-but-lacks-build-depends
W: mypackage source: package-needs-versioned-debhelper-build-depends 8
W: mypackage source: debhelper-overrides-need-versioned-build-depends (>= 7.0.50~)
E: mypackage source: temporary-debhelper-file debhelper.log
E: mypackage: no-copyright-file

The only things I added manually after that were Vcs-*, Homepage, and Enhances.

Now the only things left to do are running the thing through cowbuilder a few times and putting in all the necessary build dependencies, and writing a nice changelog entry.

Note, this method does not replace putting in some thought. But it's an interesting way to get a relatively clean package.

Enabling core files for PostgreSQL on Debian

The other day, I was a bit puzzled over a seemingly simple task: Enable core files to be generated from a PostgreSQL instance running on Debian. That instance has unfortunately been segfaulting on occasion, but never left a core file.

Now in principle it is clear that

ulimit -c unlimited

is the incantation to get this done. But where do you put this? You could hack it into the init script, but that seemed a bit ugly, and I wanted a sustainable solution.

A useful thing in the meantime is to check the current settings. That information is available in /proc/$PID/limits with the PID of the postmaster process (or any child process, really), and it looked like this to begin with:

Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            ms        
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            8388608              unlimited            bytes     
Max core file size        0                    unlimited            bytes     
...

Use sudo grep core /proc/$(sudo cat /var/run/postgresql/8.4-main.pid)/limits if you want it automated.

So it's good to know that we only need to set the soft limit.

One way to configure this properly would appear to be in /etc/security/limits.conf. There you can add a line like

*               soft    core            unlimited

to enable core dumps globally. I'm not actually sure whether that would work if the service is started during the boot without PAM. In any case, I didn't want to enable core files globally; who knows what that would lead to.

One could replace the * by a user name, such as postgres, and then enable pam_limits.so in /etc/pam.d/su. But the postgresql init script in Debian is nested about four levels deep, so it wasn't clear whether it called su at all.

Now as it turns out, the init script ends up changing the user using this Perl code:

$) = $groups;
$( = $gid;
$> = $< = $uid;

(see change_ugid in /usr/share/postgresql-common/PgCommon.pm), so the whole PAM line of thought wasn't going to work anyway. (Other packages such as pgbouncer and slony1 do got through su, so that would be a solution for those.)

The way to solve this is the pg_ctl -c option, which sets the soft limit for core files to unlimited. And the way to pass this option through the init script maze is the file /etc/postgresql/8.4/main/pg_ctl.conf, which should contain a line like this:

pg_ctl_options = '-c'

Then restart postgresql, and check /proc/$PID/limits again:

Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            ms        
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            8388608              unlimited            bytes     
Max core file size        unlimited            unlimited            bytes     

OK.

Another thing that's recommendable in this context is to change the core file names to have a unique element, so that if multiple backends crash before you can take a look, they don't overwrite each other's core files. The core(7) man page explains the configuration options; I went with this sysctl setting:

kernel.core_pattern = core.%e.%p

which includes process name and PID. The PID file still ends up in the data directory of the PostgreSQL instance, which could also be changed, but I didn't find it necessary.

Stick the above line in /etc/sysctl.d/local.conf and reload with

service procps force-reload

I actually use a setting like that on all machines now; it's just nicer.

OK, and now I'll wait for the next core file. Or not.