Moving to C++

GCC 4.8 was recently released. This is the first GCC release that is written in C++ instead of C. Which got me thinking ...

Would this make sense for PostgreSQL?

I think it's worth a closer look.

Much of GCC's job isn't actually that much different from PostgreSQL. It parses language input, optimizes it, and produces some output. It doesn't have a storage layer, it just produces code that someone else runs. Also note that Clang and LLVM are written in C++. I think it would be fair to say that these folks are pretty well informed about selecting a programming language for their job.

It has become apparent to me that C is approaching a dead end. Microsoft isn't updating their compiler to C99, advising people to move to C++ instead. So as long as PostgreSQL (or any other project, for that matter) wants to support that compiler, they will be stuck on C89 forever. That's a long time. We have been carefully introducing the odd post-C89 feature, guarded by configure checks and #ifdefs, but that will either come to an end, or the range of compilers that actually get the full benefit of the code will become narrower and narrower.

C++ on the other hand is still a vibrant language. New standards come out and get adopted by compiler writers. You know how some people require Java 7 or Python 2.7 or Ruby 1.9 for their code? You wish you could have that sort of problem for your C code! With C++ you reasonably might.

I'm also sensing that at this point there are more C++ programmers than C programmers in the world. So using C++ might help grow the project better. (Under the same theory that supporting Windows natively would attract hordes of Windows programmers to the project, which probably did not happen.)

Moving to C++ wouldn't mean that you'd have to rewrite all your code as classes or that you'd have to enter template hell. You could initially consider a C++ compiler a pickier C compiler, and introduce new language features one by one, as you had done before.

Most things that C++ is picky about are things that a C programmer might appreciate anyway. For example, it refuses implicit conversions between void pointers and other pointers, or intermixing different enums. Actually, if you review various design discussions about the behavior of SQL-level types, functions, and type casts in PostgreSQL, PostgreSQL users and developers generally lean on the side of a strict type system. C++ appears to be much more in line with that thinking.

There are also a number of obvious areas where having the richer language and the richer standard library of C++ would simplify coding, reduce repetition, and avoid bugs: memory and string handling; container types such as lists and hash tables; fewer macros necessary; the node management in the backend screams class hierarchy; things like xlog numbers could be types with operators; careful use of function overloading could simplify some complicated internal APIs. There are more. Everyone probably has their own pet peeve here.

I was looking for evidence of this C++ conversion in the GCC source code, and it's not straightforward to find. As a random example, consider gimple.c. It looks like a normal C source file at first glance. It is named .c after all. But it actually uses C++ features (exercise for the reader to find them), and the build process compiles it using a C++ compiler.

LWN has an article about how GCC moved to C++.

Thoughts?

Installing multiple PostgreSQL versions on Homebrew

I was going to post this yesterday, but some might have thought that it was a joke. April 1st is always an annoying day to communicate real information.

If you have been fond of the way Debian and Ubuntu manage multiple PostgreSQL versions in parallel, you can now have the same on OS X with Homebrew:

brew tap petere/postgresql
brew install postgresql-9.2
# etc.
brew install --HEAD postgresql-common

postgresql-common is the same code as in Debian, only mangled a little.

Now you have all the client programs symlinked through pg_wrapper, and you can use the server management tools such as:

pg_createcluster 9.2 main
pg_ctlcluster 9.2 main start
pg_lsclusters

Let me know if you find this useful.

Links:

• homebrew-postgresql
• postgresql-common homebrew branch

Lists and Tuples

I had initially found the divide between lists and tuples in Python confusing. I came from a database background, so I have a certain expectation of what a tuple might be. If you read up on what the difference is in Python, you will find that a) tuples are immutable, and b) singleton tuples use a funny syntax. So just use lists, because it's easier to read and you can't go wrong that way. Oh, and they are both sequences, another overloaded term.

(Yes, there are some details omitted here, such as that since a tuple is immutable, it is hashable and can be used as a dictionary key. But I think that is used fairly seldomly.)

Then I came across Haskell and it dawned on me: Was this just a poorly mangled feature from Haskell? I don't know the history, but it looks a bit like it. You see, Haskell also has list and tuples. Lists are delimited by square brackets, and tuples are delimited by parentheses:

let alist = [1, 2, 3]
let atuple = (1, 2, 3)

(Technically, in Python, tuples are not delimited by parentheses, but they often appear that way.) But the difference is that Haskell does not use parentheses for any other purpose, such as delimiting function arguments. It uses spaces for that. (So it looks more like a shell script at times.)

Python: len([1, 2, 3])
Haskell: length [1, 2, 3]

But in Haskell, tuples are not mutable lists and lists are not mutable tuples. Tuples and lists are quite different but complementary things. A list can only contain elements of the same type. So you can have lists

[1, 2, 3, 4, 5]
["a", "b", "c", "d"]

but not

[1, 2, "a"]

A tuple, on the other hand, can contain values of different types

(1, 2, "a")
(3, 4, "b")

A particular type combination in a tuple creates a new type on the fly, which becomes interesting when you embed tuples in a list. So you can have a list

[(1, 2, "a"), (3, 4, "b")]

but not

[(1, 2, "a"), (3, 4, 5)]

Because Haskell is statically typed, it can verify this at compile time.

If you think in terms of relational databases, the term tuple in particular makes a lot of sense in this way. A result set from a database query would be a list of tuples.

The arrival of the namedtuple also supports the notion that tuples should be thought of as combining several pieces of data of different natures, but of course this is not enforced in either tuples or named tuples.

Now, none of this is relevant to Python. Because of duck typing, a database query result set might as well be a list of lists or a tuple of tuples or something different altogether that emulates sequences. But I found it useful to understand where this syntax and terminology might have come from.

Looking at the newer classes set and frozenset, it might also help to sometimes think of a tuple as a frozenlist instead, because this is closer to the role it plays in Python.

pgindent Jenkins job

I have set up a Jenkins job that runs pgindent. Besides checking that the procedure of running pgindent works, it also provides a preview of what pgindent would do with the current source (pgindent.diff), which can be educational or terribly confusing.

Introducing the Pex package manager for PostgreSQL

I have written a new light-weight package manager for PostgreSQL, called pex. It's targeted at developers, allows easy customization, and supports multiple PostgreSQL installations.

Here is how it works:

Installation:

git clone git://github.com/petere/pex.git
cd pex
sudo make install

Install some packages:

pex init
pex install plproxy
pex search hash
pex install pghashlib

Multiple PostgreSQL installations:

pex -g /usr/local/pgsql2 install plproxy
pex -p 5433 install pghashlib

Upgrade:

pex update
pex upgrade

It works a bit like Homebrew, except that it doesn't use Ruby or a lot of metaphors. ;-)

Check it out at https://github.com/petere/pex.

PostgreSQL and Jenkins

A lot of places use Jenkins nowadays, including where I now work and have previously worked. I enjoy working with Jenkins, and so I always wanted try out how this would work with PostgreSQL. Obviously, there would be some overlap with the build farm, but that's OK. The point of the build farm, after all, is to build things in many different ways to find potential problems, and this would just support that overall effort.

So I have set this up now: http://pgci.eisentraut.org/jenkins/

It's already been very helpful during the last couple of weeks that I've run this. The main point behind the effort is to automate things. These are things I do just about every day and won't have to anymore:

• build PostgreSQL
• check for compiler warnings
• run various test suites
• do this for all supported branches

These are things I do every couple of weeks and have now automated:

• check distribution building (make distcheck)
• test build of additional documentation formats
• cpluspluscheck
• check external web links in the documentation (The job for that currently appears to be reporting false positives. Use with caution.)
• test coverage reporting

Moreover, I have set up to build some extensions and external modules, which weren't regularly tested. (The build farm is making some efforts in this area, though.)

Actually, many of the checks I had set up immediately found problems: newly introduced compiler warnings, secondary documentation format builds broken, cpluspluscheck failing, broken links in the HTML documentation, various extensions no longer build with Postg reSQL 9.3devel.

But there is more cool stuff:

• There are various RSS feeds for all builds or failed buids.
• You can interact with the system on mobile devices. I use JenkinsMobi for iOS.
• You can get up to date documentation builds on a more predictable schedule.

The one thing (just about) it doesn't do is test operating system and CPU architecture portability. Jenkins comes from a Java background, where this isn't much of an issue, and so there isn't good built-in support for that sort of thing. But anyway, we have the build farm for that.

You can get the code at http://bitbucket.org/petere/pgci. The entire setup is automated with Puppet. You can fork it and set up your own (or send me your changes), or you can run it locally using Vagrant (which is what I do to test changes).

If you have any ideas, let me know (file an issue on Bitbucket). I have plans for a number of enhancements already, foremost pg_upgrade testing. Also, let me know if there are additional extensions you want tested. I have just put in a few I use myself at the moment, but other can easily be added.

Happy New Year!

psqlrc files

In PostgreSQL 9.2, you can use major-version-specific .psqlrc files, such as .psqlrc-9.2. PostgreSQL 9.2 also added the "include relative" command \ir to psql. Combining these two, you can set up psql initialization to take advantage of any new features you want without breaking the use of old psql releases.

For example, I'd like to set up psql to automatically use \x auto. But if I just put that into .psqlrc, older psql releases will complain about an unknown command. (I usually have multiple PostgreSQL versions installed, and I share dotfiles across hosts.) On the other hand, I don't want to have to duplicate the entire .psqlrc file to add one command, which is where \ir comes in.

Here is what I use, for example:

.psqlrc-9.2

\ir .psqlrc
\set QUIET yes
\set COMP_KEYWORD_CASE preserve-lower
\x auto
\unset QUIET

.psqlrc-9.3

\ir .psqlrc-9.2