I've recently had to set up a new machine, but didn't have an install cdrom available, so I decided to use the easiest method for installing Ubuntu; PXE booting. Here's how I did it. PXE involves setting up two simple technologies, DHCP and TFTP. We start by setting up TFTP.

TFTP is Trivial File Transfer Protocol, a cut down version of FTP. There are a number of TFTP servers in Debian and Ubuntu, but not all of them support the extensions that the pxelinux bootloader used by debian-installer need. Experience has shown that tftpd-hpa works correctly, so we'll want to install that.

ace root% apt-get install tftpd-hpa

Note: If this installs an inetd at the same time, you may need to restart the inetd so it enables the tftpd service.

The tftpd will serve files out of /var/lib/tftpboot, so we need to add some files for it to serve. You can use this script to fetch various netboot installers from Ubuntu's servers.

#!/bin/bash

set -u
set -e

cd /var/lib/tftpboot

for dist in dapper feisty gutsy hardy intrepid; do
    mkdir -p $dist
    for arch in amd64 i386; do
        mkdir -p $dist/$arch/
        (cd $dist/$arch/ && ncftpget -RT \
           ftp://archive.ubuntu.com/ubuntu/dists/$dist/main/installer-$arch/current/images/netboot/)
    done
done

Download ubuntu-tftp-update.sh

Now we need to alter our dhcpd configuration. (You are using DHCP aren't you?) All we need to add is a group declaration to your subnet declaration, adding a next-server and a filename parameter. You can then add a host declaration for any machine you want to netboot into the installer.

group { # intrepid amd64
     next-server 10.0.0.1;
     filename "intrepid/amd64/pxelinux.0";
     host foobar { hardware ethernet 00:22:15:45:cc:fa; fixed-address foobar.example.com; }
}

You'll need to restart the dhcp server so it picks up the new setting. The next-server parameter is the name or IP address of your tftp server. filename is the path to the bootloader. Obviously, you can use this to pick which version of the installer you want to run. If you do a lot of installations, it might be worth configuring every installer you're likely to use and then move hosts in and out of the suitable group as and when you need to install them.

All that's left to do now is to boot the computer and set it to boot from the network and enjoy medialess installation.

If you get the following error:

/etc/ldap/slapd.conf: line 127: substr index of attribute "sambaSID" disallowed

when you run slapindex, then you haven't updated your samba.schema to the version from Samba 3.0.23. Dapper and Edgy had 3.0.22, so if you've recently upgraded to Hardy, you will see this problem. The file should have an MD5 of 0e23b3ad05cd2b38a302fe61c921f300. I'm hoping this resolves problems I have with samba not picking up group membership changes. I'll update if it does.

Update: Having installed the new schema and run slapindex, net rpc info shows I have twelve groups when previously it showed zero. This may not solve my group membership problems, but it can't be a step backwards.

Spamassassin 3.2, which is available in Gutsy and Lenny, comes with a new feature to increase performance by compiling its regular expressions using re2c. It's very quick to enable. First, you need to install the required packages:

apt-get install re2c libc6-dev gcc make

Next, edit /etc/spamassassin/v320.pre and uncomment the line that says:

loadplugin Mail::SpamAssassin::Plugin::Rule2XSBody

Next pre-compile the regular expressions using sa-compile:

femme:/etc/logcheck# sa-compile
[18741] info: generic: base extraction starting. this can take a while...
[18741] info: generic: extracting from rules of type body_0
100% [===========================] 3293.83 rules/sec 00m00s DONE
100% [===========================] 650.12 bases/sec 00m01s DONE
[18741] info: body_0: 647 base strings extracted in 2 seconds
[snip compiler output]
make install
Files found in blib/arch: installing files in blib/lib into architecture dependent library tree
Installing /var/lib/spamassassin/compiled/3.002004/auto/Mail/SpamAssassin/CompiledRegexps/body_0/body_0.so
Installing /tmp/.spamassassin18741hDrlUQtmp/ignored/man/man3/Mail::SpamAssassin::CompiledRegexps::body_0.3pm
Writing /var/lib/spamassassin/compiled/3.002004/auto/Mail/SpamAssassin/CompiledRegexps/body_0/.packlist
Appending installation info to /var/lib/spamassassin/compiled/3.002004/perllocal.pod
cp /tmp/.spamassassin18741hDrlUQtmp/bases_body_0.pl /var/lib/spamassassin/compiled/3.002004/bases_body_0.pl
cd /
rm -rf /tmp/.spamassassin18741hDrlUQtmp

Finally, restart spamassassin, and you should find it runs faster. You will need to run sa-compile every time you update your rules, or they won't take effect.

If you get the following warning:

Can't locate Mail/SpamAssassin/CompiledRegexps/body_0.pm in @INC

you forgot to run sa-compile; re-run it and the error should go away.

Apache 2.2 changed the way you configure LDAP authentication. mod_auth_ldap was replaced with mod_authnz_ldap, so don't forget to enable the new module and disable the old one. Because I'll always forget, here's the new style config.

AuthType basic
AuthName "admin"
AuthBasicProvider ldap
AuthLDAPUrl ldap://ldap.example.com:389/ou=people,dc=example,dc=com?uid?sub
AuthLDAPGroupAttributeIsDN off
Require ldap-group cn=systems,ou=groups,dc=example,dc=com
AuthLDAPGroupAttribute memberUid

The sections in bold are the sections I had to change from the 2.0 config.

Unfortunately I live in the UK, where 6 months of the year, the time is GMT. Now is the time of year when I discover which of my servers don't have the right timezone configuration and show the wrong time during daylight saving. For future reference, here's how to set the timezone to Europe/London rather than UTC.

root@cms01:/tmp/openssl-0.9.8g# date
Mon Mar 31 08:23:35 GMT 2008
root@cms01:/tmp/openssl-0.9.8g# tzconfig
Your current time zone is set to GMT
Do you want to change that? [n]: y

Please enter the number of the geographic area in which you live:


   1) Africa         7) Australia

   2) America        8) Europe

   3) US time zones     9) Indian Ocean

   4) Canada time zones    10) Pacific Ocean

   5) Asia           11) Use System V style time zones

   6) Atlantic Ocean    12) None of the above


Then you will be shown a list of cities which represent the time zone
in which they are located. You should choose a city in your time zone.

Number: 8

Amsterdam Andorra Athens Belfast Belgrade Berlin Bratislava Brussels
Bucharest Budapest Chisinau Copenhagen Dublin Gibraltar Guernsey Helsinki
Isle_of_Man Istanbul Jersey Kaliningrad Kiev Lisbon Ljubljana London
Luxembourg Madrid Malta Mariehamn Minsk Monaco Moscow Nicosia Oslo Paris
Podgorica Prague Riga Rome Samara San_Marino Sarajevo Simferopol Skopje
Sofia Stockholm Tallinn Tirane Tiraspol Uzhgorod Vaduz Vatican Vienna
Vilnius Volgograd Warsaw Zagreb Zaporozhye Zurich

Please enter the name of one of these cities or zones
You just need to type enough letters to resolve ambiguities
Press Enter to view all of them again
Name: [] London
Your default time zone is set to 'Europe/London'.
Local time is now:      Mon Mar 31 09:23:48 BST 2008.
Universal Time is now:  Mon Mar 31 08:23:48 UTC 2008.

More information is available in the Debian System Administrator Manual.

Recently, we rolled out a Shibboleth Single Sign On service to protect one of our services. However, we started recieving intermittant login failures, both on our automated monitoring and from customers. Curiously these failures tended to happen mostly in the evening, which isn't a peak time for us. Debugging showed that the authentication worked, but the authorisaton was failing. Shibboleth works as an apache module and daemom that protects a service, which communicates with a webservice that does the authenication processing. The log files were showing an occasional SSL error in this communcation link.

INFO shibtarget.SessionCache [43005] sessionGet: trying to get new attributes 
      for session (ID=_d0cd2f93840bb92050b28fa73d19ce4f)
INFO SAML.SAMLSOAPHTTPBinding [43005] sessionGet: sending SOAP message to 
      https://login.example.com/shibboleth/AA
ERROR SAML.SAMLSOAPHTTPBinding [43005] sessionGet: failed while contacting   
      SAML responder: error:1408F06B:SSL routines:SSL3_GET_RECORD:bad 
      decompression
ERROR shibtarget.SessionCache [43005] sessionGet: caught SAML exception 
      during SAML attribute query: SOAPHTTPBindingProvider::send() failed 
      while contacting SAML responder: error:1408F06B:SSL 
      routines:SSL3_GET_RECORD:bad decompression
ERROR shibtarget.SessionCache [43005] sessionGet: no response obtained

We didn't manage to find any suitable solutions on the internet, so we pulled out the trusty wireshark and started looking to see what was going on. We could see that the client was advertising deflate and null compression, and that the server was responding by asking for deflate compression. However the client would then claim that there was a decompression error in the servers response. This opened a few lines of enquiry. I made sure that both ends of the connection were running the same version of OpenSSL and they were both using 0.9.8a from Ubuntu Dapper. Interestingly 0.9.8a is the first version that had compression support. We found a couple of suggestions including forcing connections to be SSL2, which lacked compression or recompiling openssl without zlib support. As the former was easier, we tried that first by putting

SSLCipherSuite SSLv2:-LOW:-EXPORT:RC4+RSA

in /etc/apache2/mods-enabled/ssl.conf as suggested by Debian bug #338008, and this seemed to work for around an hour. Packet sniffing showed that it was still negotiating SSL3 including deflate compression. Clearly we had to try something else.

Rather than recompile OpenSSL without zlib support, I thought I'd try upgrading the version of OpenSSL to something later in case that fixed the decompression bug. the version in Hardy is 0.9.8g, which sadly required recompiling and disabling the Ubuntu change to enable -Bsymbolic-functions during linking. Installing this on the client end didn't fix the problem, however installing it on the server end seemed to fix it. So far it's been running for 24 hours without an error, so fingers crossed that this has fixed it for good.

Installing the Sun Java packages on Debian or Ubuntu require to you accept Sun's license before you can install them. This means that it's not easy to install non-interactively, for example when using pbuilder. Fortunately the license uses debconf to check to see if you have already accepted the license. This means you can use debconf to accept the license before you install the packages. Create a file containing the following lines:

sun-java5-jdk shared/accepted-sun-dlj-v1-1 select true
sun-java5-jre shared/accepted-sun-dlj-v1-1 select true
sun-java6-jdk shared/accepted-sun-dlj-v1-1 select true
sun-java6-jre shared/accepted-sun-dlj-v1-1 select true

Then run /usr/bin/debconf-set-selections <file> and when you install the java packages, you should find it doesn't prompt for the license any more.

By default, warnquota sends out emails with the device name in the message, which probably doesn't make much sense to most non-technical users.

Hi,

We noticed that you are in violation with the quotasystem
used on this system. We have found the following violations:


/dev/mapper/Ubuntu-home

                        Block limits               File limits
Filesystem           used    soft    hard  grace    used  soft  hard  grace
/dev/mapper/Ubuntu-home
               +- 1044404 1000000 1200000  6days    1781     0     0

You can improve this by using /etc/quotatab to assign a more meaningful name to the partition:

/dev/mapper/Ubuntu-home:user directory
/dev/mapper/Ubuntu-shared:shared area

I've been running Hardy on my workstation for a while and had recently noticed that I was failing to type a space after "I". I was doing it far too much for it to just be me failing to press the space bar properly, and it wasn't happening after any other letter. After a little bit of experiementing, I discovered that something was eating shift-space. What was happening was that I was failing to release the shift key quick enough after typing "I" and before I hit the space bar, so it wasn't getting passed on.

Turns out that the problem was a recent update of Hardy installed SCIM, which uses shift-space as a keyboard shortcut. To turn it off, load the SCIM Setup program and go to the FrontEnd Global Setup screen and remove "Shift+Space" from the Trigger hotkey.

Caused confusion for a few minutes. :)

dpkg has a very useful feature where if you delete a conffile (pretty much everything under /etc and a few other files) it isn't replaced when you upgrade the package[0]. This behaviour was confusing me for a while until I realised what was happening. I was attempting to reinstall a package to get the default configuration files back that had been accidentally deleted, but no matter what I tried, the files didn't exist after running dpkg. Once I figured out that dpkg had this behaviour the solution was simple; use the --force-confmiss command line argument.

root@quux:~# dpkg --force-confmiss -i /tmp/foo_2.0.0-build.14_all.deb 
(Reading database ... 33418 files and directories currently installed.)
Preparing to replace foo 2.0.0-build.14 (using .../foo_2.0.0-build.14_all.deb) ...
Unpacking replacement foo ...
Setting up foo (2.0.0-build.14) ...

Configuration file `/etc/foo/foo.xml', does not exist on system.
Installing new config file as you request.
root@quux:~#

[0] If the file didn't exist in the previously installed version, it is installed, so you get new configuration files.

mojo-jojo david% ant clean
Buildfile: build.xml

clean:
     [echo] Deleting all project files
   [delete] Deleting directory /home/david/tomcat5.5-5.5.20/servletapi
   [delete] Deleting directory /home/david/tomcat5.5-5.5.20/container
   [delete] Deleting directory /home/david/tomcat5.5-5.5.20/jasper
   [delete] Deleting directory /home/david/tomcat5.5-5.5.20/connectors
   [delete] Deleting directory /home/david/tomcat5.5-5.5.20/build

BUILD SUCCESSFUL
Total time: 46 seconds
mojo-jojo david% ls
build.xml  debian/
mojo-jojo david% 

*blink*

I installed awstats today for the first time on my personal domain and reading though the search terms is always amusing, but I got concerned by people finding my site searching for "bonnie langford naked" and "naked bonnie langford". Turns out I was talking about Bonnie Langford in a post about Charlie and the Chocolate Factory and the naked thing was to do with body scanners on the London Underground.

But what really worried me was the search for "scott james remnant naked".

About a year ago I had a problem with udev crashing during startup on my powerpc box. Somehow I managed to muddle on with this problem, probably by not rebooting the box. :) Last summer I had to reoot it again so I did a bit more research and discovered that udev was trying to looking up the nvram group, not finding it in /etc/group and then trying ldap, which, of course, failed because we have no networking yet.

Adding the group fixed the bug and filed a bug against udev saying that udev should add any groups it used. Carrying out further debugging revealed that the crash was during nss_wins. The general order of events were:

1. udev looks up a user or group.
2. Group doesn't exist in compat.
3. Lookup in ldap.
4. Ldap attempts to resolve the name of the ldap server or client. (server is 127.0.0.1 so confused about this point.)
5. Network and/or dns server isn't up so dns fails fails.
6. Attempts to look up host in wins.
7. udevstart crashes.

I didn't have time to debug this any further and proceeded to forget the problem, but last night my fileserver started having the same problem. Removing ldap from passwd, group and shadow resolved the udev problem, but then I didn't have any users. Late last night I booted without ldap and then changed nsswitch.conf to add ldap, and went to bed.

This morning I had an epiphany in the shower. Not only did I remember what the bug was, but also a sensible workaround. The problem wasn't with the passwd et al lines, but the hosts line. I did have

hosts: files dns mdns wins

The solution is to return if dns isn't available and changed the line to:

hosts: files dns [UNAVAIL=return] mdns wins

Now all I need to do is to debug nss_wins and get to the bottom of the crash. It might be worth filing a bug against nss_ldap for trying to do a lookup against an ip address.

Having just mistakenly performed a large upgrade including powerdns, I'm wondering if there isn't something we can do to minimise the amount of time we stop a service between pre-inst and post-inst. Could we have something like a post-unpack maintainer script or a flag telling dpkg that the package contains a daemon and to minimise the time between the maintainer scripts where possible.

Yes I should read the list archives, but I'm about to go out. Must remember to check when I get back.

Edit:Adeodato Simó pointed out that in most cases daemons should just do a restart in the postinst, unless changing files on disk is a problem. Need to investigate if this is an issue for pdns-server and fix it.

Finally have irssi-0.8.10-rc6 packages out. Hopefully 0.8.10 will be out soon so I can upload it to debian without using nasty version numbers. The biggest change from the version in sid is that I've changed the package name to irssi and it should replace both irssi-text and irssi-snapshot. It also includes a devel package, so I'd appriciate if people could test upgrading and building irssi plugins. Email me any bugs you find.

deb http://www.davidpashley.com/debian/irssi ./

Yesterday was a little more productive. Finally got round to starting merging irssi-text and irssi-snapshot into a irssi package. Need to do silly little things like putting files into the right packages, and then I can start merging some of the patches from 0.8.9-9.

Finland appears to be playing havoc with my hayfever. Typically I forgot to being anything and the chemist was shut. A very nice lady answered my plea for an anti-hystamine, but I don't think it completely prevented me feeling like crap. Also woke up to find that I'm being eaten alive by mosquitos. Bastards. Weather should be 25C today, so a few degrees lower than the last two days, but higher humidity. I was expecting hot weather, but this is silly

Not too much done yesterday. Mostly bitching about the temperature and faffing on irc. Last night was mostly spent drinking beer and plotting world domination.

Scott, Phil and Vince do not look happy this morning.

I'm in Helsinki. This is good.

I'm awake. This is not so good

Left Laura's flat yesterday at 1pm and proceeded to visit pretty much every bomb site via the northern line, to drop her off at Euston so she could return to her parents and then on to Paddington by bus to meet Robot101. Rather surprisingly we managed to meet up without too much hanging around and then catch the slightly slower, but much cheaper Heathrow Connect. The train is on the departures board as being to "Hayes & Harlington. Also stops at Heathrow." despite heathrow being beyond Hayes. It might as well have added "but don't tell any one". I suspect Heathrow Express might have words if they advertised going to Heathrow too much. I really don't think it is worth paying 14.00GBP over 9.50GBP for the ten minutes less it takes for the non-stop service.

Met up with various other people at Heathrow consisting of Scott James Remnant, Steve McIntyre, Rob Taylor, Phil Hands and Simon Kelley. Flight delayed by 25 minutes, although it was about 18:40 befre we actually took off. Arrived in Helsinki around midnight, only to find it doing some impression of dusk. Surprised to be met by Jesus and other locals so we didn't need to work out the busses, even though various people had been organised enough to print out the lovely photographic guide to HUT.

It's now 8am localtime so time for breakfast. It has no right being this bright, this early in the morning.

I have recently added a patch to irssi-text to fix #274201 (Displays incorrect Chinese characters on input bar). As I don't use Big5 myself, I can not properly test the patches. I would appriciate it if anyone that does use Big5 can install http://www.davidpashley.com/debian/irssi-text/irssi-text_0.8.9-6_i386.deb and let me know if it works for them.

Finally managed to get my ipv6 transport working and gain access to a IA64 box running Debian (well Ubuntu, but close enough) to debug the FTBFS on itanium (Thanks Thom). Turns out that the Ubuntu package didn't build either and after several aborted attempts to play around with the static specifier, I noticed that debian/rules special-cased not using -O2 on ia64. Removed that and it now compiles on both at least i386 and ia64. Hopefully my sponsor will build on powerpc and alpha for me too.

irssi-text (0.8.9-3) unstable; urgency=medium

  * Fix linker errors on IA64 by building with -O2 on that platform too.
  * Upload with medium urgency to get the previous fixes into sarge

 -- David Pashley <david@davidpashley.com>  Sat, 19 Mar 2005 17:43:07 +0000

I also spent an hour or two this morning tracking down a bug when using a turkish locale. It turned out that the perl plugin was taking the name of the different chat networks (e.g. IRC, SILC) and lowercasing all but the first character using g_strdown() to give the name of the perl module (Irssi::Silc). The problem was that g_strdown() uses the current locale to do the lowercasing and in turkish, the lower case of I is not i. A quick change to use the newer g_ascii_strdown() and irssi now starts in a turkish locale.

--- irssi-text-0.8.9.orig/src/perl/perl-common.c
+++ irssi-text-0.8.9/src/perl/perl-common.c
@@ -565,8 +565,8 @@
        chat_type = chat_protocol_lookup(rec->name);
        g_return_if_fail(chat_type >= 0);

-       name = g_strdup(rec->name);
-       g_strdown(name+1);
+       name = g_ascii_strdown(rec->name,-1);
+       *name = *(rec->name);

        /* window items: channel, query */
        type = module_get_uniq_id_str("WINDOW ITEM TYPE", "CHANNEL");

I have asked the submitter to check this bug and will upload once 0.8.9-3 is in sarge. I also need to send this patch upstream. In the meantime if you use a turkish locale, can you test out the package available at http://www.davidpashley.com/debian/irssi-text/

irssi-text (0.8.9-4) unstable; urgency=low

  * Correctly lower case chat protocols using g_ascii_strdown() rather than
    using the deprecated g_strdown() (Closes: #232628)

 -- David Pashley <david@davidpashley.com>  Wed, 23 Mar 2005 08:29:32 +0000

Finally got round to updating irssi-text packaging and fixing a few bugs that would be nice to get into sarge if I can. You can find a copy on my website. Currently in need of a sponsor, hint hint.

 irssi-text (0.8.9-2) unstable; urgency=medium
 .
   * Upload medium as it would be useful to get the irssi-common and perl fixes
     into sarge.
   * Conflict with irssi-common (Closes: #263320)
   * Update to Standards-Version 3.6.1
   * Update to building against libglib2.0
   * Fix the menu file quoting
   * Depend on autotools-dev and update config.{status,guess} automatically
     (Closes: #296989)
   * Recompile against libperl5.8 (5.8.4) (Closes: #248020, #224930, #247104)

Update:Steve McIntyre has sponsored an upload of i386, powerpc and alpha for me

I gave up reading the hot-babe thread shortly after it descended into discussing the crusades. To me, the whole discussion has failed to mention the more interesting question of whether the package is useful and if we want Debian to be full of silly pointless packages. I know traditionally if someone was willing to maintain a package and it was DFSG-free then it accepted into the archive. But now sarge doesn't even fit on a DVD anymore we have to start asking ourselves "Just how many media players/irc clients/load meters/menstrual calendars do we need in Debian?" Maybe it is time someone went through the archive, looking at the orphaned, unmaintained and trivial packages and ask "Do we really need this package?"

I installed a new machine this week. A nice little Shuttle box which should be fairly quiet, except the heatsink on my graphics card need replacing. I did the usual method of installing XP and then installed Debian using a sid D-I business card CD. I must say D-I on i386 is fantastic. I set it all up using a 2.6 kernel and configured LVM, which while long winded, worked perfectly. Rebooted into Debian and had a working machine. The installer detected my XP installation and configured Grub accordingly. So far so good.

Then I rebooted into XP.

GRUB showed the commands it had executed but then it just hung. The XP startup screen didn't appear. Nothing. I could reboot into Debian fine and could mount my NTFS partition and see all my data. Reboot. same problem. Loaded the XP rescue tools from the CD and fixed the mbr. As I'd over written grub, the machine just hung at boot.

I put this down to XP having issues with LVM and reinstalled both XP and Debian on the machine, this time without LVM and putting grub on /dev/hda2 rather than /dev/hda. It did the same thing. Argh.

After a bit of research, I discovered that problem was down to disk geometry. A forum led me to a support article from SuSE about the exact same problem. Fortunately, setting my bios to LBA rather than Auto fixed it. Further searching produced a few more articles and reports of Mandrake suffering for the same problem. It turns out that the problem only occurs iff you install using a 2.6 kernel and your XP partition is not completely in the first 8.6GB of the drive. In 2.6, the kernel developers decided that discovering the correct disk geometry was a userspace task. When the drive is partitioned the kernel reports the untranslated geometry, so an incorrect partition table is written. When XP boots it uses BIOS calls to find the geometry, which reports a translated geometry, which no longer matches what is in the partition table, so it hangs. Changing the BIOS is a workaround, but fixing the partition table is the correct fix.

I'll file a bug as soon as I work out where the correct place to file it is. I'm currently thinking partman. Below is some further reading on the problem.

• Bill Moss
• O'reilly
• O'reilly
• Slashdot

Last night I successfully managed to install Debian on my AlphaStation 255, after 5 attempts. d-i in sid is almost there. Had a couple of niggles. It seems that something broke the code to detect if the drive was using BSD disklabels. vorlon gave me points of where to look to fix it and 30 minutes later I had a fix. Hacking d-i in d-i is cool. Having stuff written in bash is very useful. Shame it doesn't have a full vim installation :). I need to check if the bug with the partitioner not detecting that you have /boot on a separate partition is still there.

The only other issue I had was that you need to leave a small space at the start of the drive for the bootloader. vorlon neglected to tell me this until I tripped over it (grrr :) ) but a quick repartition and a (not so) quick base install later and it was up and running.

I celebrated with a beer and some chocolate.