Tricky shell local variables

I have a word of warning against improper use of local in shell functions.

If you are using shell functions, you might want to declare some variables local to the shell function. That is good. The basic syntax for that is

local a b c

In some shells, you can also combine the local declaration and assignment, like this:

local foo=$1
local bar=$2

(The Debian policy even explicitly allows it.)

This is somewhat dangerous.

Bare shell assignment like

foo=$bar

does not perform word splitting, so the above is safe even if there are spaces in $bar. But the local command does perform word splitting (because it can take multiple arguments, as in the first example), so the seemingly similar

local foo=$bar

is not safe.

This can be really confusing when you add local to existing code and it starts breaking.

You can avoid this, of course, by always quoting everything to like

local foo="$bar"

but overquoting isn't always desirable, because it can make code less readable when commands are nested, like

local foo="$(otherfunc "other arg")"

(Nesting is legal and works fine in this case, however.)

I suggest using local only for declaring variables, and using separate assignment statements. That way, all assignments are parsed in the same way.

Tracing shell scripts with time stamps

A random tip for shell script hackers. You know that with set -x you can turn on tracing, so that every command is printed before being executed. In bash, you can also customize the output prefix by setting the PS4 variable. The default is PS4='+ '. Here is an example. I wanted to "profile" a deployment script, to see why it took so long. Ordinarily, I might have sprinkled it with date calls. Instead, I merely added

set -x
PS4='+\t '

near the top. \t stands for time stamp. (The script was already using bash explicitly, as opposed to /bin/sh.) That way, every line is prefixed by a time stamp, and the logs could easily be analyzed to find a possible performance bottleneck.

pipefail

It is widely considered good style to include

set -e

near the beginning of a shell script so that it aborts when there is an uncaught error. The Debian policy also recommends this.

Unfortunately, this doesn't work in pipelines. So if you have something like

some_command | sed '...'

a failure of some_command won't be recognized.

By default, the return status of a pipeline is the return status of the last command. So that would be the sed command above, which is usually not the failure candidate you're worried about. Also, the definition of set -e is to exit immediately if the return status of the last pipeline is nonzero, so the exit status of some_command isn't considered there.

Fortunately, there is a straightforward solution, which might not be very well known. Use

set -o pipefail

With pipefail, the return status of a pipeline is "the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands exit successfully". So if some_command fails, the whole pipeline fails, and set -e kicks in. So you need to use set -o pipefail and set -e together to get this effect.

This only works in bash, so if you're trying to write scripts that conform to POSIX or some other standard, you can't use it. (There are usually other ways to discover failures in pipelines in other shells, but none are as simple as this one, it appears.) But if you are writing bash anyway, you should definitely use it. And if you're not using bash but use a lot of pipelines in your scripts, you should perhaps consider using bash.

(Hmm, it looks like there could be a number of latent bugs in the existing Debian package maintainer scripts, because this issue appears to be widely ignored.)