Solid-State Drive Benchmarks and the Write Cache

Like many people, we are interested in deploying solid-state drives (SSDs) for our database systems. Jignesh posted some performance test results a while ago, but as I had commented there, this test ran with the write cache on, which concerned me.

The Write Cache

Interlude: The disk write cache is the feature that causes you to lose your data when the server machine crashes or loses power. Just like the kernel may lie to the user-land application about writing stuff to disk unless the user-land application calls fsync(), the disk may lie to the kernel about writing stuff to the metal unless the write cache is turned off. (There is, as far as I know, no easy way to explicitly flush the cache. So write cache off is kind of like open_sync, if you know what that means.) As PostgreSQL pundits know, PostgreSQL does fsyncs at the right places unless you explicitly turn this off, and ignore all the warning signs on the way there. By contrast, however, the write cache is on by default on consumer grade ATA disks, including SATA disks and, as it turns out, also including "enterprise" SSD SATA devices.

To query the state of the write cache on a Linux system, use something like hdparm -W /dev/sda. To turn it off, use hdparm -W0 /dev/sda, to turn it back on, hdparm -W1 /dev/sda. If this command fails, you probably have a higher-grade RAID controller that does its own cache management (and doesn't tell you about it), or you might not have a disk at all. ;-) Note to self: None of this appears to be described in the PostgreSQL documentation.

It has been mentioned to me, however, that SSDs require the write cache for write wear leveling, and turning it off may significantly reduce the life time of the device. I haven't seen anything authoritative on this, but it sounds unattractive. Anyone know?

The Tests

Anyway, we have now gotten our hands on an SSD ourselves and gave this a try. It's an Intel X25-E from the local electronics shop, because the standard, big-name vendor can't deliver it. The X25-E appears to be the most common "enterprise" SSD today.

I started with the sequential read and write tests that Greg Smith has described. (Presumably, an SSD is much better at being better at random access than at sequential access, so this is a good worst-case baseline.) And then some bonnie++ numbers for random seeks, which is where the SSDs should excel. So to the numbers ...

Desktop machine with a single hard disk with LVM and LUKS over it:

• Write 16 GB file, write caching on: 46.3 MB/s
• Write 16 GB file, write caching off: 27.5 MB/s
• Read 16 GB file: 59.8 MB/s (same with write cache on and off)

Hard disk that they put into the server that we put the SSD in:

• Write 16 GB file, write caching on: 49.3 MB/s
• Write 16 GB file, write caching off: 14.8 MB/s
• Read 16 GB file: 54.8 MB/s (same with write cache on and off)
• Random seeks: 210.2/s
  

This is pretty standard stuff. (Yes, the file size is at least twice the RAM size.)

SSD Intel X25-E:

• Write 16 GB file, write caching on: 220 MB/s
• Write 16 GB file, write caching off: 114 MB/s
• Read 16 GB file: 260 MB/s (same with write cache on and off)
• Random seeks: 441.4/s
  

So I take it that sequential speed isn't a problem for SSDs. I also repeated this test with the disk half full to see if the performance would then suffer because of the write wear leveling, but I didn't see any difference in these numbers.

A 10-disk RAID 10 of the kind that we currently use:

• Write 64 GB: 274 MB/s
• Read 64 GB: 498 MB/s
• Random seeks: 765.1/s

(This device didn't expose the write cache configuration, as explained above.)

So a good disk array still beats a single SSD. In a few weeks, we are expecting an SSD RAID setup (yes, RAID from big-name vendor, SSDs from shop down the street), and I plan revisit this test then.

Check the approximate prices of these configurations:

• plain-old hard disk: < 100 €
• X25-E 64 GB: 816.90 € retail, 2-5 weeks delivery
• RAID 10: 5-10k €

For production database use, you probably want at least four X25-E's in a RAID 10, to have some space and reliability. At that point you are approaching the price of the big disk array, but probably pass it in performance (to be tested later, see above). Depending on whether you more deperately need space or speed, SSDs can be cost-reasonable.

There are of course other factors to consider when comparing storage solutions, including space and energy consumption, ease of management, availability of the hardware, and reliability of the devices. It looks like it's still a tie there overall.

Next up are some pgbench tests. Thanks Greg for all the performance testing instructions.

(picture by XaYaNa CC-BY)

Do you have a good idea for PostgreSQL?

Do you have a feature request or a good idea for how to improve PostgreSQL? Subscribing to a mailing list is too cumbersome? Or you did subscribe to a mailing list and presented your idea there, and now it's rotting in the archives or on the todo list? Let's try something different.

I have set up a feedback forum over at UserVoice. There you can enter your ideas, comment and vote on other suggestions, and see what the top requests are, without the overhead of tracking a mailing list every day. Let's consider this an experiment. It is not going to replace the existing project communication channels, and you shouldn't send bug reports or engage in coding discussions there. But the more people raise their "user voice" there and provide useful suggestions and comments, the more useful it might become.

To try it, go to: http://postgresql.uservoice.com/

The floodgates are open for development on PostgreSQL 8.5, so now is the time to make yourself heard.

(Disclaimer: UserVoice is a commercial company. I am not associated with them. This is just an attempt to find better interfaces for user feedback.)

(picture by NoNo^Q8 CC-BY)

Where have all the translations gone?

If you have downloaded PostgreSQL 8.4.0 and are wondering where so many of the translations have gone: The translation team has decided not to ship translations anymore that are not translated at least about 80%. (See the commit message for the list of victims.) This is so that incidental users of stale translations are not presented with a confusing and distracting mix of translated and untranslated messages all the time. So right now we are only shipping a full or almost full set of translations into German, Spanish, French, Japanese, Portuguese, and Turkish.

To get the translations into other languages back into the release, go to http://babel.postgresql.org/ and start submitting updates. Updates may be included as early as release 8.4.1 in a few months.

I hope in particular that we might get the Chinese, Italian, and Russian translations back into shape.

By the way, if you want to start (or continue) translating, I suggest that you approximately follow this priority order: libpq, psql, pgscripts, pg_dump, initdb, postgres. This or a similar order will make the translations useful to the most users with the least amount of work.

10+5 Things (Not Yet) Rendered Obsolete By My Mobile Phone

I don't have an iPhone, but I figured when reading about the 15 Apps Rendered Obsolete By The New iPhone 3GS that while my mobile phones have over time obsoleted several gadgets, tools, and other non-phone items, there are still a lot of things they don't do for me, but conceivably should. At the moment, I have a Nokia E71, which is a pretty good phone. But here are a few things it could do better:

1. Calendar. I still carry a separate small pocket calendar for my personal and business appointments. Mostly because I haven't bothered to figure out how to synchronize the phone calendar with the calendar at the office. And then, I don't think it's easily possible to sync my work appointments with the calendar system at work but back up my personal appointments to MyNokia Backup (or is it Ovi now), without sending the somewhat confidential work appointments to Nokia and the somewhat confidential private appointments to the office. I think you can exclude certain events from being synced at all, but not sync certain events only to certain servers. And as a minor point, the phone calendar doesn't know the local holidays. The calendar issue is obviously a core competency of the mobile phone, so it might just be matter of time before it's solved. Best guess: End of the year, when I need to decide whether to buy another calendar book.
2. Notes. The phone has a Notes application (and an Active Notes application, whatever). But there's the synchronization issue again; how do I get the notes from the phone into my action and project lists (which I don't keep on the phone). I do use the phone to take notes on the train and during meetings, say, but then I transfer them manually to a more trusted container.
   
3. GPS receiver. I keep a separate GPS receiver from Garmin for geocaching. The E71 has a GPS receiver and a maps application, and I do use it for navigation in the car, and I have recently discovered the SportsTracker application. But the simple interface of entering coordinates and having an arrow point there that the Garmin provides does not appear to be available on the phone. I do know that some people go geocaching with only their phones, so this must be possible, but I haven't had time to figure it out yet. But this gadget might become the next one to go.
   
4. Flashlight. This one ties in with a recent geocaching experience. Many people have tried to use their mobile phone as a light source in a bind and noticed that it doesn't work so well. Perhaps they could put in a few LEDs and a battery that is two millimeters thicker. That would help a lot.
5. Laser pointer. If they can do flashlight, they can surely do laser pointer. Useful for presentations, when you don't want to carry around an extra gadget.
6. Bluetooth mouse. Also for presentations, how about just using the phone as a next-slide clicky thing. This is likely already possible with some custom software on both the phone and the laptop, but it could be easier.
7. Camera. I'm not an expert photographer by any means, and I haven't done any great analysis of the picture quality of the built-in camera, but it just feels better to take a separate digital camera when you actual go out with the purpose of taking pictures.
8. USB drive. Yeah, you can do this with the cable and then taking care to copy things into the right directory on the phone's memory card or something. Could be easier and clearer, though. And separate from the phone's internal file system. Putting a full-size USB plug into the phone is probably going to be difficult, but for larger models like the Nokia E90 it could be possible.
9. Debian. Nothing like being able to hack up your own operating system and applications as you choose. It's sad that the Openmoko/FreeRunner initiative has had setbacks. And while Android is an open platform, there is something to be said for having the same operating system on all devices.
10. Desktop computer. Yeah right. Most of the "productivity" applications on the phone still suck and are at best stopgaps until you get home to the big screen. More on that in a later post.
    

OK, those weren't 15 things, as with the iPhone. For completeness, how about 5 things that the phone does appear to have replaced permanently:

1. Address book. I used to have one, but all the current addresses of relevance are in the phone, backed up to somewhere on the Internet. In fact, I could probably find most people I know either via public mailing list archives or something like Facebook anyway.
   
2. Land line phone. The stationary phones are gone for good. There is now only this one phone, which is the home phone, the work phone (also no more desk phones at the office), and the VoIP phone. I found this review of Google Voice an interesting contrast in this context. I only have one phone number anyway. (The VoIP line has a different number, but there you want the separate number to make sure you are using the cost-saving path to call.)
   
3. Car satnav system. For the few times I have needed it, the satellite navigation system in the phone has done fine. It's not quite the same as having one built into the car, in terms of ease of use, GPS reception, and integration with the other audio equipment, but it works. And you can use the same system to also navigate on walks, as opposed to drives, which is a surprisingly useful feature.
   
4. MP3 player. I have never really listened much to MP3s anyway, and now on the phone I have podcasts, video podcasts, internet and broadcast radio, as well as MP3s, which is much better altogether.
5. iPhone. :-) Don't have one, don't want one. I recommend the Hacking the iPhone session from 25C3 (video page) for further enlightenment.
   

Weltkulturerbe weg, Kultur bleibt

So, jetzt ist es also passiert. SPIEGEL ONLINE berichtet, dass die UNESCO Dresden den Weltkulturerbetitel aberkannt hat, wegen der Waldschlösschenbrücke. Ist vielleicht besser so, denn jetzt kehrt hoffentlich Ruhe ein. Man konnte die ewigen Diskussionen ja nicht mehr hören.

Man kann die Brücke ja gut oder schlecht finden. Was ich aber nicht nachvollziehen kann ist Folgendes. Das Dresdner Elbtal war ja auf der Liste des Weltkulturerbes, nicht etwa des Weltnaturerbes; das gibt es auch. Kultur also im Sinne von vom Menschen Erschaffenes, nicht nur Musik und Malerei oder sowas. Das, was die UNESCO als schützenswert auserkoren hatte, waren also nicht die schönen grünen Elbwiesen, sondern das Elbtal von Pillnitz bis Übigau, mitsamt Parks, Gebäuden, Brücken, Werften und Schiffen, so die Begründung für die Aufnahme. Die Begründung erwähnt, dass verschiedene Errungenschaften vom 16. bis zum 20. Jahrhundert im Dresdner Elbtal versammelt sind. Warum muss aber die Kultur im 21. Jahrhundert aufhören? Die Menschheit entwickelt sich schließlich weiter, und die Zeichen, die diese Entwicklung setzt, können und sollen auch in eine außerordentliche Kulturlandschaft einfließen. Im Dresdner Elbtal wurden schon diverse Brücken gebaut, die teilweise wie das "Blaue Wunder" selbst Sehenswürdigkeiten sind und sogar ausdrücklich von der UNESCO erwähnt sind. Die neue Waldschlösschenbrücke ist vielleicht nicht die schönste Brücke der Welt, aber sie ist ein Ausdruck der Kultur der Gegenwart.

Der andere Aspekt, der diese Geschichte zur Farce machte, ist das Lamentieren der hohen Politik und nationalen Prominenz. Fakt ist: Die Dresdner Bürger haben in einer Volksabstimmung für die Brücke gestimmt. Und selbst nachdem die UNESCO angefangen hatte, ernsthaft damit zu drohen, den Titel abzuerkennen, haben in mehreren Umfragen deutliche Mehrheiten gemeint, dass sie trotzdem wieder für eine Brücke stimmen würden. Warum das so ist, sei dahingestellt. Vielleicht ist den Dresdnern der schnelle Weg zu Arbeit oder zum Flughafen wichtiger. Oder man hat erkannt, dass das kulturell wertvolle Blaue Wunder eigentlich ein Verkehrsnadelöhr ist. Aber die Bürger haben entschieden und das gilt es zu akzeptieren. Die Bürger können ja später auch mal dafür stimmen, die Brücke wieder abzureißen.

Praise to LXDE

I've been know to rant at times about crappy Linux desktop software. So here's a praise for a change: a praise to LXDE.

You know when the name of a piece of technology starts with "Simple" or "Lightweight", it's really not? SMTP, SNMP, LDAP? Well, LXDE proves that wrong.

I first heard about LXDE from Klaus Knopper's presentation at Chemnitzer Linux-Tage 2009, where he described why he chose LXDE as the new default desktop for his well-known Knoppix live CD/DVD/environment.

I have never really been interested in the debate of late about how short we can possibly get the boot time of a Linux machine/desktop, since I've been using suspend-to-disk for years everywhere. I only reboot for kernel upgrades, and possibly when the wakeup gets botched, which happens about once a month, I guess. But that's a different matter. What I was interested in was a lightweight desktop environment to run inside a VirtualBox.

So LXDE. apt-get install lxde gives you the whole thing. Well, the only thing that's missing is a web browser, which you will have to select and install yourself. I had reported previously on the amusing quest to find a "lightweight" web browser.

Unlike some geek-enabled minimal desktop environments, LXDE doesn't surprise the average user with an unusual layout. You have a taskbar at the bottom (can be moved to the top, for those used to GNOME), with a menu button, buttons for file manager, terminal, browser, minimize all windows; on the right, there is a CPU meter, a clock, a screen lock and a logout button. At least on Debian, the terminal and browser buttons call x-terminal-emulator and x-www-browser, respectively, so whatever browser you choose to install, it will work. There is also an image viewer and a handful of minor tools and settings available through the menu. And you can start random commands with Alt+F2. But that's more or less it.

Another thing that is interesting about LXDE is that it makes "ps" useful again. Under KDE, a "ps x" on relatively idle desktop produces what feels like 50 processes. Under LXDE it shows 15, which includes the terminal, bash, and ps processes to produce the listing. And it looks like with a bit of effort that number could be reduced even further.

Even though it's quite small, LXDE supports freedesktop standards. It uses desktop files, dbus, openbox as window manager, and supports compiz if you want. You can easily run GNOME or KDE applications, and they will behave reasonably. Once you do, however, they will of course load their lot of libraries and daemons, and the lightweightness will be gone. So on a workstation desktop, where you might want to run a graphical mail program, a calendar application, network manager, update notifier, and so on, LXDE will probably not buy you much. But on netbooks and virtual boxes, where all you need is a browser and a shell, this is a great alternative that is usable by everyone, and one that is true to its lightweight attribute. So far.

SlideShare and Creative Commons

Today I uploaded all the presentations that I have held at tech conferences over the years to SlideShare. (At least those that were in a reasonable format. I was a user of DocBook Slides in the early days, which produces HTML, which you can't upload to SlideShare.) That was quite a trip through history for me; 6 years of traveling, meeting people, pondering ideas.

I have also gone through and licensed all presentations and this blog under a Creative Commons license. For various reasons I chose different variants of Creative Commons for different bits, but in the future I plan to mostly use the CC-BY variant, which is much like the BSD license in spirit.