Had an interesting problem today with one of our servers at work. First thing I noticed was yesterday that an upgrade of Apache2 didn't complete properly because /etc/init.d/apache2 stop didn't return. Killing it and starting apache allowed the upgrade to finish. I noticed there was a zombie process but didn't think too much off it.

Then this morning got an email from the MD saying that various internal services websites were down (webmail, wiki etc). My manager noticed that it was due to logrotate hanging, again because restarting apache had hung. Looking at the server I noticed a few more zombie processes. One thing I'd noticed was that all these processes had reparented themselves under init and a quick web search later confirmed that init(1) should be reaping these processes. I thought maybe restarting init would clear the zombies. I tried running telinit q to reread the config file, but that returned an error message about timing out on /dev/initctl named pipe. I checked that file existed and everything looked fine. The next thing I checked was the other end of the named pipe by running lsof -p 1. This showed that init had /dev/console rather than /dev/initctl as fd 0. I tried running kill -SIGHUP 1, but that didn't do anything. Then I tried kill -SIGUSR1 1, but that didn't do anything either. I checked the console, but there wasn't enough scrollback to see the system booting and decided to schedule a reboot for this evening.

Rebooting the server presented me with an interesting challenge. Normally the shutdown command signals to init to change to runlevel 0 or 6 to shutdown or reboot using /dev/initctl. Of course init wasn't listening on this file, so that was out. Sending it an SIGINT signal (the same signal init gets on ctrl-alt-delete) had no response. Obviously telinit 0 wasn't going to work either. I decided to start shutting services down manually with the help of Brett Parker. The idea was to stop all non-essential services, unexporting nfs exports, remounting disks read-only and then either using sysrq or a hardware reset. Unfortunately someone accidentally ran /etc/init.d/halt stop, hanging the server, but he is suffering from a bad cold today so I forgive him. The server restarted without a hitch (thank god for ext3) and running lsof -p 1 showed init having /dev/initctl open. I don't know what happened to init the last reboot on Monday, but a reboot seemed to fix it. Odd bug, but thankfully it was a nice simple fix. I could have spent the evening debugged init. :)

I'm currently mourning the loss of my laptop's hard disk. I don't think there was massive amounts of data on there that I needed, but it's still upsetting. Looks like I'll have to buy a new 2.5" drive. At least it gives me a reason to reinstall Debian.

Update: The bad news is that it is actually a 1.8" drive, which means I need to spend 93GBP including tax and postage for a new drive. Should get it by Friday. The good news is that having left the laptop off during the day, I managed to get the laptop booting and am not rsyncing the data off as fast as I can. Shame about the money, but at least I haven't lost much in the way of data.

I'd like to announce the initial release of Eddie, a feed parser library written in Java. It's taken me over 100 hours, but it now correctly parses 90% of the FeedParser unit tests, including all the rss and atom tests. It's GPLed, with an exception allowing you to use it in any open sourced program. Get it at my website. Need to add documentation and character set and encoding support. Also need to separate the testing infrastructure from the rest of the code.

This is the first time I've done any java programming in anger, and I have to say I'm surprised to discover I quite like it. In many ways it seems a very quick language to program in. It seems almost like programming in a scripting language, but stronger typed. This is probably due to not having to worry about memory management. Certainly I don't think I could have written this quite so quickly in C++.

Having said that, there are a couple things that I don't like about Java. Everything is a pointer. This is useful at times, but it means that every time you want to call a method on an object you have to test whether it is null or you run the risk of getting the dreaded NullPointerException. Java also doesn't have keywords for and, or and not. I know not everyone likes these, but I keep finding myself trying to use them.

I'm sure there are other things I hated, but I can't remember them now. I think I'll end up doing more java programming in the future.

To the anonymous Texan that thought it would be a good idea to point out a Perl module in response to my date parsing class in Java:

I'm well aware of date parsing modules in other languages. In fact I've used them for similar tasks. But the post wasn't asking "How do I parse dates in Perl?" It was giving some code that some people might find useful.

Commenting with "man Date::Parse" doesn't make you look clever; it just makes you look like a twat and the kind of person people are relucant to invite to parties. You may be a geek and you may know stuff, but that doesn't mean you have to try to look clever, because you will invariably fail.

</rant>

Oh and Erich, I've got some more date formats I want to add support for, so I'll add german dates and post the updated code. :)

I've recently had cause to parse some date values in Java. As a result I've produced a class which can manage to parse an awful lot of date formats. I thought I'd better document it in case someone found it useful. Certainly there doesn't appear to be anything elsewhere which shows you how to parse lots of formats. I have found the order of date_formats to be very brittle, so I don't recommend you change it without an awful lot of test cases.

Anyway, without further to do, I present to you, the Pathological Date Parser for Java

// Copyright 2006 David Pashley <david@davidpashley.com>
// Licensed under the GPL version 2
import java.text.SimpleDateFormat;
import java.util.Calendar;
import java.util.TimeZone;

public class Date {
    private Calendar date;

    static String[] date_formats = {
            "yyyy-MM-dd'T'kk:mm:ss'Z'",        // ISO
            "yyyy-MM-dd'T'kk:mm:ssz",          // ISO
            "yyyy-MM-dd'T'kk:mm:ss",           // ISO
            "EEE, d MMM yy kk:mm:ss z",        // RFC822
            "EEE, d MMM yyyy kk:mm:ss z",      // RFC2882
            "EEE MMM  d kk:mm:ss zzz yyyy",    // ASC
            "EEE, dd MMMM yyyy kk:mm:ss",   //Disney Mon, 26 January 2004 16:31:00 ET
            "-yy-MM",
            "-yyMM",
            "yy-MM-dd",
            "yyyy-MM-dd",
            "yyyy-MM",
            "yyyy-D",
            "-yyMM",
            "yyyyMMdd",
            "yyMMdd",
            "yyyy", 
            "yyD"
    
    };
    public Date(String d) {
        SimpleDateFormat formatter = new SimpleDateFormat();
        d = d.replaceAll("([-+]\\d\\d:\\d\\d)", "GMT$1"); // Correct W3C times
        d = d.replaceAll(" ([ACEMP])T$", " $1ST"); // Correct Disney timezones
        for (int i = 0; i < date_formats.length; i++) {
           try {
              formatter.applyPattern(date_formats[i]);
              formatter.parse(d);
              date = formatter.getCalendar();
              break;
           } catch(Exception e) {
              // Oh well. We tried
           }
        }
        
    }
}

The only date formats I can't get it to parse are <4-digit year>-<day of year> and <2digit year><day of year> (e.g. 2003-335 and 03335 for 2003-12-01). If you can add support for those and other date formats I'll gladly take patches.

Have you ever wanted to call a member function in your class, but not known what it will be at compile time? I'm writing a SAX parser and would like a function for every element name. I could write a massive switch statement in the startElement function, but this will quite quickly become unmanagable for a large schema. The alternative is to look to see if a particular member function exists and call it.

To do this little bit of magic we need to use Java's introspection API. The first thing to do is to get a Class object for our class. We can do that by calling:

Class klass = this.getClass();

We can then look up the method we are looking for using Class.getMethod, but this function requires an array of types that the method we are looking for takes as parameters, so we get the right version of an overloaded method. We can do this with:

Class[] arguments = { Int.class, String.class, URL.class};
Method method = klass.getMethod("foo", arguments);

Now we have our method, we can call it using the Method.invoke() call. This takes an object as the first parameter, which we can use this, and an array of Objects for the parameters.

Object[] values = {bar, baz, quux};
method.invoke(this, values);

But what happens if our class has no member method called foo()? Well, Class.getMethod() will throw a NoSuchMethodException, so we can just throw a try/catch block around the code to deal with unhandled functions. It's worth pointing out that Class.getMethod() also throws SecurityException and Method.invoke() throws IllegalAccessException, IllegalArgumentException and InvocationTargetException, so you'll want to catch Exception too.

We can chain some of these calls together and the result for my SAX parser is:

public void startElement(String uri, String localName, String qName, Attributes atts) 
            throws SAXException {
   try {
       Class[] argTypes = { String.class, String.class, String.class,
               Attributes.class };
       Object[] values = { uri, localName, qName, atts };
       this.getClass().getMethod("startElement_" + localName, argTypes)
               .invoke(this, values);
   } catch (NoSuchMethodException e) {
       log.debug("unhandled element " + localName);
   } catch (Exception e) {
       e.printStackTrace();
   }
}

With this arrangement, when I want to handle a new element in my code I can just make a function like:

public void startElement_foo(String uri, String localName, String qName, Attributes atts)
            throws SAXException {
   ... 
}

Another article for your viewing pleasure. This article describes how to use Perl's IO based IO::Handle IO system and a couple of modules that allow you do to some interesting things like seamlessly handle compressed files and calculate MD5 sums as you read a file in.

use IO::Digest;

my $fh = new IO::file($filename, "r");
my $iod = new IO::Digest($fh, ’MD5’);

read_and_parse($fh);

print $iod->hexdigest

I hate backticks. They have no place in modern shell programming. Here is a couple of reasons why this is the case.

Not nestable

backticks, by their nature, are not nestable. You can not have a command expansion inside another with back ticks.

command `foo ` bar ` baz`

Should the shell expand foo and baz or bar and then foo <output of bar> baz? As it happens it will run the first one. Using the modern command expansion syntax you can write:

command $(foo $( bar ) baz)
command $(foo ) bar $( baz)

Backticks are invisible

The backtick symbol is too small and too easily confused with a single quote to be used for writing maintainable code. The alternative is significantly larger and therefore more obvious.

ls "'`!!`'"
ls "'$(!!)'"

Here is a zoomed version of the line above in my terminal:

Please consider using the bracketed version of command expansion rather than backticks and make the world a nicer place

Ever wanted to know who was logged into your oracle server and where from? This SQL will show you the username connected, which machine they are connected from and what time they connected.

SELECT s.username, s.program, s.logon_time 
   FROM v$session s, v$process p, sys.v_$sess_io si 
   WHERE s.paddr = p.addr(+) AND si.sid(+) = s.sid 
   AND s.type='USER';

Since upgrading to GNOME 2.14, I have been revisited by an annoying problem with gnome-terminal. Gnome-terminal sets your character encoding to being the same as your locale by default, which unfortunately was being detected as ANSI_X3.4-1968, while I had my $LANG set to en_GB.UTF-8 in my ~/.bash_profile. The reason it wasn't being detected was because nothing between logging in and starting gnome-terminal looked at that file, so gnome-terminal thought the locale was C.

The result was corrupt display when programs attempted to display unicode characters. I could fix it by changing the character encoding using the menu, but I'd have to do this for every tab, which quickly becomes annoying. Time to find a fix.

Turns out that you need to tell gdm to set the right locale, which you can do by configuring ~/.dmrc. Mine now looks like:

[Desktop]
Session=gnome
Language=en_GB.UTF-8

Obviously, the important section is the Language line. You need to set it to a locale that exists on your system, which you can find using locale -a. Once you've set that and logged in again, everything should be working correctly.