So I have set this up now: http://pgci.eisentraut.org/jenkins/
It's already been very helpful during the last couple of weeks that I've run this. The main point behind the effort is to automate things. These are things I do just about every day and won't have to anymore:
- build PostgreSQL
- check for compiler warnings
- run various test suites
- do this for all supported branches
- check distribution building (
make distcheck
) - test build of additional documentation formats
cpluspluscheck
- check external web links in the documentation (The job for that currently appears to be reporting false positives. Use with caution.)
- test coverage reporting
Actually, many of the checks I had set up immediately found problems: newly introduced compiler warnings, secondary documentation format builds broken, cpluspluscheck failing, broken links in the HTML documentation, various extensions no longer build with Postg reSQL 9.3devel.
But there is more cool stuff:
- There are various RSS feeds for all builds or failed buids.
- You can interact with the system on mobile devices. I use JenkinsMobi for iOS.
- You can get up to date documentation builds on a more predictable schedule.
The one thing (just about) it doesn't do is test operating system and CPU architecture portability. Jenkins comes from a Java background, where this isn't much of an issue, and so there isn't good built-in support for that sort of thing. But anyway, we have the build farm for that.
You can get the code at http://bitbucket.org/petere/pgci. The entire setup is automated with Puppet. You can fork it and set up your own (or send me your changes), or you can run it locally using Vagrant (which is what I do to test changes).
If you have any ideas, let me know (file an issue on Bitbucket). I have plans for a number of enhancements already, foremost pg_upgrade testing. Also, let me know if there are additional extensions you want tested. I have just put in a few I use myself at the moment, but other can easily be added.
Happy New Year!
Actually, Peter, you can do cross platform testing with Jenkins using a build slave on the target architecture / os. You need a separate job for each one, though - it doesn't handle combinations or minor variations of the same job very well, whereas the buildfarm handles this right out of the box.
ReplyDeleteGlad to see people in the PG world using Jenkins. We use it a lot for app development and testing, including apps which talk to postgres.
We use Jenkins for PostGIS for buildinging the official documentation,doxygen, regress against various PostgreSQL versions 9-9.3dev as well for building PostGIS Windows experimental builds and windows 64-bit production builds and general regression testing. I am interested in incorporating PostGIS pg_upgrade tests into the mix sometime as well as other OS -- just doing Debian 64-bit and Windows mingw at moment.
ReplyDeleteJenkins is doing great for cross platform compat testing on our internal Jenkins; I'm using it to test 32-bit and 64-bit on Linux and Windows via build slaves and a matrix/multi-configuration build.
ReplyDeleteWith label filters and user-defined axes you can do a lot; for example, I'm using the same slave for the 32-bit and 64-bit windows builds by creating a slave label axis named "TARGET_ARCH" with both "x86" and "x64" as values and calling "setenv /%TARGET_ARCH%" to get the target arch from the environment.
I'd love to chat about this, as it's clear you've been doing some interesting work on C++ support, warnings handling, etc, and I've been doing a fair bit with portability and multiple slaves. Maybe we can work together to help us both get a more comprehensive setup up and running?
With the multijob plugin you can also easily chain together steps like "build source tarball from git", "build binary" and "test on platform matrix".
ReplyDelete