Saturday, July 11, 2009

Adding Color to the Console: Code Syntax Highlighting with less and source-highlight

Syntax highlighting is a standard feature of every nontrivial editor, but if you just want to look at a file or the output of a command, then you will often only stumble upon niche solutions such as colordiff, or you end up using vim as your file viewer. But one very flexible program that apparently doesn't get much attention is source-highlight, available from the GNU project or as a Debian package. Besides being able to syntax-highlight all the usual programming and markup languages, it can also produce various output formats such as HTML, LaTeX, and of course plain text.

The standalone usage is a bit bizarre, because it tries to produce HTML by default and write the output to a file named infile.html. To produce colored plain text on standard output, use something like
source-highlight -fesc -oSTDOUT FILENAME

To get the most out of this, integrate it with the less pager program, so all the files you look at are automatically colored. Create a file .lessfilter in your home directory with the following contents:
#!/bin/sh

source-highlight -fesc -oSTDOUT "$1" 2>/dev/null
and make this file executable. Be sure to redirect the error output away as shown, so that you don't get annoying messages when source-highlight complains in case it doesn't know the format of the file. Then put the following in your shell startup file (.bashrc or .zshrc):
export LESS="-R"
eval "$(lesspipe)"
The first line makes sure that less will show colors, and the second line sets things up so that the .lessfilter file is invoked.

That's it. It recognizes the file type by the file name extension, so try anything like
less somefile.c
less somefile.py
less somefile.patch
to see the colors. It won't work when less is at the end of a pipe; I have some ideas on how to work around that, which I will post here later.

26 comments:

  1. Extremely cool! Thanks! I had to add "-i" before "$1" on Ubuntu Jaunty.

    ReplyDelete
  2. Only the first few lines of your post shows up on planet.debian.org. You may want to look into that.

    ReplyDelete
  3. Awesome! I confirm that it needs -i on Ubuntu.
    Plus, if you use --failsafe, you shouldn't need to redirect stderr

    ReplyDelete
  4. Well,
    please keep in mind that adding lesspipe to less will slow down operations on large files a lot.
    When I am operating on big log files (for me few GB means big) jumping to the end (G) without lesspipe is instant, whereas after enabling lesspipe takes several seconds. Actually my lesspipe is much more trivial than source-highligt, so I would expect delays much more noticeable. Note that for smaller files (like most source codes, html pages or something) things will probably work as a charm. In my case I would probably create shell script (named probably pretty-less.sh), which would invoke source-highlight, then pass it to less -R. Not as automatic as described solution, but would suit my needs better.
    Regards,
    Robert

    ReplyDelete
  5. Why don't you use 'view' for viewing text files? :)

    ReplyDelete
  6. JFTR: It's broken on Debian Unstable

    ReplyDelete
  7. BooXteR - if you ask me - because according to my knowledge view doesn't offer as good support for files changing on-the-fly (like logs which I usually analyse with less). I haven't found anything better than less.

    ReplyDelete
  8. Another tool (that I believe supports more file formats?) is Pygments. On Debian it provides the "pygmentize" tool that be used just like source-highlight.

    However, it's written in Python, so it may be a little slower than source-highlight, written in C++.

    ReplyDelete
  9. "Why don't you use 'view' for viewing text files? :)" - another reason is that vi(ew) can't cope with large files. It probably tries to load whole file into memory and it doesn't work for bigger sizes.
    Robert

    ReplyDelete
  10. @Sam,

    Ubuntu has a new major upstream version, so perhaps the invocation was changed. I will investigate that.

    ReplyDelete
  11. @Anonymous,

    There's me trying to do the world a favour by not spamming the feeds with superlong articles, and within minutes someone complains. :-) So now it's back to the full article.

    ReplyDelete
  12. @Anonymous/Robert,

    I guess I don't look at that many large files. It could be a problem, I admit.

    ReplyDelete
  13. @BooXteR,

    I don't think "view" really accomplishes the same thing. For example, viewing html, pdf, or odt with "view" shows the text, not the source. Of course you can switch to raw, but it's not the default. Also, it doesn't actually use any colors, when I try it.

    Of course the main problem is that "less" is hard-wired into my fingers. :-)

    ReplyDelete
  14. Hi
    I'm glad you're using source-highlight :-)
    source-highlight already ships a script (and some documentation) to work with less:
    http://www.gnu.org/software/src-highlite/source-highlight.html#Using-source_002dhighlight-with-less

    ReplyDelete
  15. @betto,

    Yeah, I think I saw that when I was first trying this out, but somehow didn't like the solution so much. I think it's more or less equivalent, though.

    ReplyDelete
  16. If I want to look up a shell file, foo, and its name doesn't end with .sh, less doesn't highlight it. All working fine if filename is ended with .sh. So, is it possible to add a default option? Something like: all files are shell files although their filenames are not ended with .sh

    Thanks.

    ReplyDelete
  17. @Luca Invernizzi,

    I would advise against the --failsafe option. The trick is that if the lessfilter exits with a nonzero status, then lesspipe itself will handle the file, and that includes niceties that such automatically unpacking gzip and archive files. If you use source-highlight --failsafe, then lesspipe will assume that the lessfilter has succeeded in its job, and you will see the garbled literal gzipped (or whatever) content.

    ReplyDelete
  18. If you use --infer-lang option, and the shell script starts with the #!/bin/sh then source-highlight will highlight it as a shell script no matter what its extension is (you may want to take a look at the file src-hilite-lesspipe.sh distributed with source-highlight.
    Hope this helps :-)
    Lorenzo

    ReplyDelete
  19. Hello betto. Thanks for your help.
    In Debian stable / testing / unstable the source-highlight versions are 2.x and I think that don't support --infer-lang option (it doesn't appear in man page). So, I have developed a "patch":
    #!/bin/sh

    FILE="$1"
    EXTENSION=${FILE##*.}
    NAME=${FILE%.*}

    if [ "$NAME" = "$EXTENSION" ]; then
    EXTENSION=""
    fi

    if [ "$EXTENSION" = "" ]; then
    source-highlight --src-lang=shell -fesc -oSTDOUT "$1" 2>/dev/null
    else
    source-highlight -fesc -oSTDOUT "$1" 2>/dev/null
    fi

    ReplyDelete
  20. It looks like Debian package is pretty old... I always suggest to download the .tar.gz source from the GNU site and compile it (as long as you have installed the libboost_regex dev package from Debian, the compilation is smooth)

    ReplyDelete
  21. Thanks much for the info.
    Using the info here, and especially at Lorenzo Bettini's source-highlighting page on gnu.org, I simply used the following to do syntax-highlighting in less:

    export LESS=" -R "
    export LESSOPEN="| /home/ali/bin/src-hilite-lesspipe.sh %s"

    where /home/ali/bin/src-hilite-lesspipe.sh is just my own trivial mod of
    /usr/share/source-highlight/src-hilite-lesspipe.sh
    to have it skip my large readme files:

    case $source in
    *readme)
    ;; # NOP

    Otherwise, the large, text files do slow down excessively.
    The above works for even cscope.

    Now I finally understand why 'less' has not incorporated this feature! :-)

    thanks,
    Ali

    ReplyDelete
  22. Alternatively, you can use less.vim, a pager with less keybindings but with the color syntax highlighting of vim:
    http://huyz.us/2011/a-less-like-pager-with-color-syntax-highlighting/

    ReplyDelete
  23. An additional tip is to put

    zrun source-highlight ...

    into the .lessfilter file, to handle compressed files as well.

    ReplyDelete
  24. did you ever get this to work with less at the end of a pipe? I've wasted most of my morning trying to figure this one out, lol

    ReplyDelete
  25. I've never used that at the end of a pipe, and that was not the original idea... :)

    ReplyDelete