What's it all about, Alfie?

While reading up on my newsgroups this morning an interesting thread came across my path in one of the SGI newsgroups. It was talking about the GTK+ themes and how the default GTK look-and-feel is so much different than the Irix default window manager. Well... as it turns out, someone at SGI decided to take it upon themselves to make a GTK theme that made the GTK apps - gEdit, gimp, etc. look like they fit in the Irix window manager. The theme engine is called Buffy and has several pre-built themes corresponding to the different color schemes in Irix's 4Dwm. Interesting...

Now I'm no look-and-feel bigot. I like a lot of different UIs - in fact, there probably isn't one that I dislike, but I have to agree that the GTK apps just seem to stick out - and that's not optimal. So I decided to give this a try. Very interesting and very easy to get to work. Simply build it and then put the appropriate theme gtkrc in your home directory as .gtkrc and it's ready to roll.

No functionality change here, just a little more consistent look for the apps that use GTK. Nice.

I put in a new filter into CIA for Joel yesterday... basically, his idea is to have a user pick a representative pixel and then based on some quantization of that pixel (Hue, Red, etc.) create a filter based on that quantization centered around the value for that pixel. Nothing that earth-shaking, but it's certainly a nice addition to CIA. I put it in, and sent the code up to the web site where, it appears, Joel runs his data through. Interesting.

With the problems CIA is having, I'm really wondering if I need to hide it on the CVSweb display. I mean really... who's looking at this stuff? I've suggested it to more than a dozen people, but it doesn't seem to matter. I'd show the web site I built for Leslie, but that contains sensative information that I think we shouldn't have running loose in the world. But CIA is a good example of C and I might just open it up...

I decided that today might be an interesting day to start digging into PGP and making it available for folks to send email to me encrypted. So I did a Google search and then hit on the MIT PGP page and got the code seemingly working fine from within Outlook 98. Not too bad. The only problem is the issues of how to get/accumulate public keys from other folks, and how to distribute my public key.

Interestingly, the easiest thing they could have said is to run PGPkeys... that did everything I need. I generated a key pair of the right length, with my own passphrase, and then sent it to the PGP public key server. Very clean, very nice - but also very non-obvious. Had they put one little phrase on the web site, or in the downloaded docs, it would have saved me quite a bit of time.

Anyway... now I have the PGP stuff ready to roll and it's clear how to get public keys for other folks from the servers. Nice. Don't quite know that I'll need to use it, but it's nice to have learned a little bit more about this today.

I was thinking that since I had most of GNOME built, I'd get gnumeric going as well... well... that turned out to be a significant addition. Here's what I had to do to get it going:

• gperf - from the GNU mirrors, it's a nice hash function (so they say).
• gb - GNOME Basic which is of marginal use in that gnumeric doesn't work with all versions, and so this can really be left out without any real loss.
• libglade - this was an upgrade as I needed the latest version.
• glade - ditto - upgrading one means getting the latest of the other.
• libole2 - linking and embedding library.
• libxslt - maybe not necessary, but since I was there getting stuff, I got it.
• libgtop - profiling library for GNOME.
• gtop - app to profile a system.
• Error 0.13 - this is a perl module needed for ORBit's CORBA::ORBit interface.
• CORBA-ORBit - this is a perl interface for ORBit.
• GNOME-GNORBA - this is a higher level interface for GNOME and ORBit.
• gnome-python - this is the python interfaces to GNOME.
• gnome-objc - these are the Obj-C interfaces to GNOME.
• gnome-utils - these are general utils from the GNOME collection.
• libgda - the GNOME Database Adaptor for generic database access.
• gnome-db - the Access-like tool for GNOME databases.
• gnumeric - the spreadsheet.

Maybe if I have time I'll look into writing CORBA plugins for gnumeric just to see how it goes together.

Today I went downtown for an interview with Bank of America. It took several hours, and here's hoping that I hear from them tomorrow with good news. I could really use some good news.

I got word from Leslie on the WebPipe.net hosting and moved the files up in preparation for the hosting. Looking very good... They left out the information about the PostgreSQL database - how to connect to it, accounts, etc. so I had Leslie ask them. Once I get that I'll be able to go in and set up the database and get the site rockin'. Then it's a simple matter of telling Network Solutions to use a different DNS for naturalmeat.com and it'll be live. Very nice to be so close to getting it working.

I thought it might be nice to get gEdit running on sparky and since I had built ORBit and gonme-libs for another tool, I thought it would be relatively easy to get it built. Boy, was I wrong. But in the end, it was a good thing to do.

The building took a few days and most of this was seeing what had to be accomplished and then doing it. Lots to do and no place that wrote down the build order. Too bad.

• bzip2 - while you can get this from SunFreeware, it doesn't include the libraries and the headers that are needed. Build it as both static linked and dynamic for greatest flexibility.
• guile - easy to build not much to say about it.
• popt - useful library as I've come to see, and it's pretty easy to build.
• indent - useful pretty-print routines for ORBit and many other tools - used as an additional service.
• ORBit - essential CORBA GNOME services.
• Oaf - services built upon ORBit and popt.
• GConf - important libraries for GNOME.
• gnome-libs - useful stuff and pretty low-level. Not many dependencies here.
• gnome-print - printing utilities for GNOME, very low-level.
• gnome-common - common docs and a few scripts, not nearly as much as one might think.
• libxml - important XML libraries.
• libxml2 - second generation of the above library.
• bonobo - component model built on ORBit.
• libunicode - low-level unicode support for GNOME.
• gal - the GNOME Application Library - quite useful and essential for GNOME tools.
• gtkhtml - a GTK HTML widget that seems pretty nice, but as a sister tool in gnome-core.
• scrollkeeper - a documentation tool.
• gnome-vfs - virtual filesystem support for GNOME.
• control-center - essential libraries and goodies for GNOME including the CApplet library.
• gnome-core - the essential applications and applets for the GNOME desktop.

After all these are built and installed correctly, the system is ready to roll... you can get the gEdit source and it builds cleanly and works like a charm. I did most of this on Friday and the in the evenings over the weekend. I'm glad it's done as it bring sparky to almost a complete GNOME system along with my other recent Unix machines which is nice. gEdit is fast and doesn't show a significant load on the machine, so it's nice to have gone through the trouble to figure out the dependencies and built everything for this box.

This morning we had a problem with the weather scraping off the Weather Channel site. It seems they have done something that makes the wget 1.5.x fail on the grab. Interestingly, the getPage program I wrote based on the GNOME libghhtp library works fine. Wow... I'm impressed. Well... the latest wget 1.6 works fine and so for sparky I've simply removed the old wget I got from Sun Freeware and grabbed the source for wget 1.6 from the GNU archives. I built it and it works fine. On tux I've simply gone back to using my version and it works fine.

Interesting... very interesting.

On a somewhat related note I've been working for a while on a new feature for CIA. When I 'paint' a region I use a recursive function that does the job. Well... in some cases the recursion migh go very, very deep. On the CellAnalysis web site machines it causes a core dump. So, what I wanted to do was to un-'dig' the routine and make it an iterative one. Very similar to a loop for factorial as opposed to recursion. Well... it didn't quite work out that way, but almost as nice. The trick was to implement a looping algorithm that utilized a stack to save the points I was visiting, and in doing that we achieved what was desired - minimal system stack usage.

Interestingly enough, the routine is faster in that there aren't the overhead calls, and it uses less memory since only the relavent values are on the stack, so it's a pretty good win all around. Sure, the complexity goes up a little, but that's a reasonable trade-off for not having to worry about stack implosion.

Today I spent some time migrating the CIA Uploading and processing code to PHP 4.04pl1 because there were differences that I hadn't taken into account in 4.04, and I wanted to get the code working again as Joel was having so much trouble getting his PHP code to work. So I did it. I haven't liked that BMP files don't display properly in Netscape, but that's another story - and in fact, I created a wonderful work-around in CIA that solves yet another problem - BMP file size.

You see, the original scripts planned on having only JPEG files. But they are lossy and BMPs are not. So CIA can read either and convert from one to another. So if you give CIA a BMP file it will generate a JPEG file so that the scripts can use it for presentation. The fact is that JPEGs are a lot smaller than BMPs, so even if they send a BMP we want to show them a JPEG. It'll display faster, and that's important.

So now no matter what is uploaded, a JPEG is generated for viewing purposes. This is a good step forward. Additionally, the comment Joel made about problems in PHP 4.04pl1 are not true - at least not generally true. I have use the PHP 4.04pl1 source and built it and it shows no signs of adding additional lines to the beginning of a file. None at all.

On the syrup front, it looks like my cousin will be going with Online Merchant Gold a package that does a complete eCommerce site for you for a nominal set-up fee and a recurring $70/mo with 4.95% of each sale. This is a reasonable set-up because it offers the Certificate and SSL which would be at least $35/mo and then there's the support you get and the database and software.... so it's a reasonable deal. They do all the card processing sending you orders and checks. Sounds nice.

I also got the easyDNS working with my domain TheManFromSPUD.com and converted all my machines to this new domain name. Very nice to have everything more stable and working well. I haven't tried sendmail, but now that I have DNS, I'll be able to do that if I so choose.

I've also taken the time to update all the web pages to remove the IP addresses and put in the machine names with the domain so I don't have to worry about updating those pages if/when the IP addresses from MediaOne change. I have to say that this is one of the better things I've done in the office for a while. It's nice and clean and cheap, and it's great to know that people can find my machines no matter where they are. Cool!

UPDATE: there's another eCommerce solution that Leslie is considering. Their present web hoster has a simple eCommerce option that might just do the trick. She's checking into it to see how they do card processing and shipping, but it sounds pretty interesting for the price.

This morning I had finished downloading all the kernel RPMs for 2.2.19-6.2.1 and could take the time to upgrade the kernels of both mao and tux. It was really quite easy and painless. The only problem is that because I keep tux up so much, I had to try and remember what I had running after the reboot. Not a big problem, but a pain.

I spoke way, way too soon...

DISASTER! When I did the upgrade to tux I didn't count on the fact that the upgrade wouldn't be able to find his Compaq Smart Array 3200 controller. The upgrade on mao worked so well I didn't even give it a second thought. But when I tried to reboot tux I got no SCSI devices found and the machine did a kernel panic.

I was agast with chest pains.

If the system can't boot, then there's no way to see the drives and have any chance of fixing the problem. I tried building a boot floppy on mao only to later realize that it wouldn't work because a boot floppy assumes the system set-up at hand. This means that I couldn't possibly build one for tux as he wasn't running. No, I didn't make one before the upgrade - which I had in the past, but will now for certain.

The problems continued when I finally got the RedHat 6.2 rescue mode off the install CD-ROM. The rpm program on the CD was based on an older version (v3) and the RedHat advisory had me upgrade the database so it's incompatible with the one on the CD.

So... In order to fix the problem, I had to be on the drive, mounted as root with everything working OK. Sounds impossible? I thought so too. But here's what I did to fix it.

1. Boot off the RedHat 6.2 CD and enter Rescue Mode by typing linux rescue at the opening prompt. Thankfully, this version of the kernel sees the SCSI array and loads the drivers but doesn't create any devices in /dev.
2. With the rescue system up, look at the file /proc/partitions and see that they have the partitions as they should be. This requires knowing how your machine was set up, but that's reasonable in this situation. What you need to get are the Major and Minor numbers for all the partitions on the disk you have. We'll use them to manually create devices in the next step.
3. At the prompt, type the following commands - one for each partition:

   # mknod /dev/sda b 72 0
   # mknod /dev/sda1 b 72 1
   # mknod /dev/sda3 b 72 3
   # mknod /dev/sda5 b 72 5
   # mknod /dev/sda6 b 72 6
   # mknod /dev/sda7 b 72 7
   

   where in this example the Major number was 72 and the Minor numbers were 0, 1, 3, 5, 6, 7. It's important to understand the size and how Linux is laid out. For my installation, and may others, there is a /boot partition, a swap partition and then the rest of the drive is
   /. For me these are the last three, in order.

4. Now we need to make a directory to mount this filesystem. I use /d via:

   # mkdir /d
   

   and then you mount the root filesystem there with:

   # mount -t ext2 /dev/sda7 /d
   

   At this point we're getting close to having something useful. We still need to mount the /boot partition properly with:

   # mount -t ext2 /dev/sda5 /d/boot
   

5. Now cd to /d and see if your files are there. They should be. If not, then you have even bigger problems than I did - good luck. But if you see them, then you need to get the old kernel RPMs back onto your system and 'downgrade' to get back to working. The trick here is that ftp needs to have the network working and /etc/services defined and it isn't in the rescue partition. To configure the network interface I used:

   # ifconfig eth0 24.29.224.2 netmask 255.255.255.0 broadcast 24.29.224.255
   # route add -net default gw 24.29.224.1
   

   where I knew the address that I needed to use and the gateway as well.
   Next use pico, or another editor to create a minimal
   /etc/services:

   ftp	21/tcp
   

   Now you'll be able to fun ftp. A word of warning: use the '-n' option on ftp and the USER command because in this minimal configuration ftp can't understand the automatic login responses from the server.

6. OK... we're almost done. Now we need to make this mounted filesystems look like root so that we can run the existing rpm and fix the system. That's done with:

   # chroot /d
   

   where once again, the mount point for the drive's root is /d.

7. Now we can run any command as if the system were up and happy. This is a major breakthrough as it means that you only have to have the RedHet CD to fix a problem as severe as this one. I did a:

   # rpm -ivh kernel*.rpm --force
   

   and while it complained as usual, it worked. I then did the mkinitrd for the two kernels and it worked perfectly.

tux is back, but it took me more than 24 hrs. to figure this out. This should be a FAQ on the RedHat site, but it isn't. At least I have it here now.

One of the things I'd like to get into this journal is the basics of upgrading the Linux kernel so that it's written down in a good place to find it. Here goes...

1. Get the RPMs for the kernel. Find out which ones you need by doing an
   rpm -qa | grep kernel and then removing the entries for
   kernelcfg and the like. For me this includes -src, -doc,
   -headers, -pcmcia, -source, -utils and for one machine -smp. Just make sure you match what your system has.
2. If you want make a back-up floppy you can do that as root with:

   # /sbin/mkbootdisk --device /dev/fd0 2.2.14-12
   

   where the version (2.2.14-12) is obtained from /etc/lilo.conf and is pretty self-evident. The important issue is that this is the current version of the kernel.

3. Upgrade all the kernel RPMs as root with:

   # rpm -Uvh kernel*.rpm
   

   there may very well be errors due to non-empty directories. This is not a problem as those directories that could not be removed can easily be removed manually.

4. For SCSI-based machines, create new RAM Disk images as root with:

   # /sbin/mkinitrd /boot/initrd-2.2.16-3.img 2.2.16-3
   # /sbin/mkinitrd /boot/initrd-2.2.16-3smp.img 2.2.16-3smp
   

   and remove any old .img files for previous versions of the kernel.

5. Now edit /etc/lilo.conf and change the references to the old kernel version to the new version. This will most likely be in the image= and initrd= lines. To make these changes effective in the system you need to run lilo as root:

   # /sbin/lilo -v
   

6. Reboot.

These are pretty terse, but they'll get you through an upgrade.

Yesterday I received yet another notice about an ISP that allowed MySQL/PHP/Apache services, but not PostgreSQL. I wanted to find out what I didn't know about MySQL so I started looking into it on barney which has it installed.

Well... I certainly know why they like it. The entire database is file-based. This makes for easy accounting - doesn't it. Since an ISP wants to limit the entire diskspace per client, it makes sense that a database that's file-based is a good thing. Well... the fact is, PostgreSQL is also file-based, but the ownership of the files is that of the process running the database. Is it possible to still calculate the total used space? Yes. But it's more difficult, that's true. So now it's clear why. Let's see what it costs us.

PostgreSQL is very much like any of the high-end commercial packages. Here are a few things that MySQL doesn't have that PostgreSQL does:

• Foreign Keys - this is almost hard to believe. They even say in their documentation that this is a good thing as FKs are difficult to deal with. Now I'm no database snob, but I can't believe that anyone who maintains business critical data thinks this. FKs are the key to referential integrity and without that you have no guarantees that the data you have is related to the key components for it's understanding. Imagine an order item with $1,000 without an order to go with it? Do you assume it's bad data, or do you try and find the customer that's missing a $1,000 item, or find where your receivables are? This is the kind of thinking that I simply can't believe they didn't do.
• Stored Procedures - if you can parse SQL - which they do, then you should be able to parse scripts. If you can do that you can handle stored procedures. There's nothing to it. While it may not be the most efficient language to use, a few additions to straight SQL make it very reasonable for SPs. I'm really amazed that they don't consider this to be a useful addition. But looking at their attitude towards FKs, this doesn't surprise me.
• Transactions - for a database to be SQL-based, I'm amazed that such a fundamental concept can be thoughtfully left out. While I can certainly understand that there are plenty of applications that don't need transactions, there really aren't many that are business critical that don't. This then places the target market for MySQL squarely at the web server, text logs, non-business critical database market. There really isn't any other way to look at all these omissions other than to realize that there is a different market for this database.
• Triggers - this is a little easier to believe because a lot has to go into handling triggers. You ought to have a transaction to have a trigger so you can roll it back if the trigger senses that something's gone wrong.

Combine this with the fact that PostgreSQL is every bit as fast as MySQL and you're in the position of choosing a solid, complete database versus a table/record database. I can't even image why someone you willingly choose MySQL. I can understand why certain shops don't offer PostgreSQL, but that may be as much education as anything else. We'll have to see.

I got word from the ISP for North Hollow Farm that the account was set up for the shopping cart experience. I went and checked it out. I also checked into the UPS tools like address verification, time tracking and package pricing and services. There is good news and bad news about each, I'm afraid.

Most importantly, the shopping cart is reasonably well done with just enough customization to be nice, while still making it wasy for a new person to use. This will be important because there's a lot of data to load/type in, and while I can physically do it, she might as well do it because she knows it, and the maintenance practice will do her good. The bad news is that the order flow is fixed, and there's no way to interject the UPS tools into the flow if they aren't done by the software manufacturer. This isn't really surprising to me, but it's unfortunate.

On the UPS software, even if we could have interrupted the order flow, there's a problem with the UPS tools that I hadn't counted on - XML. The documentation said that the tools were available in HTML and XML forms, but that's only true for the most simplistic of tools. If we needed to address verification or time tracking, then we'd have to have XML, and that means CGI at least, and that's a server configuration issue. Not a pretty sight, I promise you. So... it's probably just as well that the order flow can't be modified - I'm not sure we could have modified it anyway.

This brings me to an idea... I wonder if I should just do it myself and then see how it compares. The issue really isn't anything other than what's best for Leslie. I can't really do card processing, but I can get an interface and try... I haven't done SSL/HTTPS, but I can find out about it and try... I guess it comes down to having the time to try versus doing something else.

I think I'll wait and see what Leslie says. Depending on me for code is not necessarily in her best interests.

I was doing a little reading yesterday and have to say that I'm more than a little disappointed in the state of JSP engines like Tomcat. If you look at similar scripting engines like PHP or even Microsoft's ASP, the web server is smart enough to know what to do based on the file extansion. But not Tomcat. It requires not only the file extension, but also the location. This is what I find a real pain.

I'm going to look into it more today to see what I can accomplish. Maybe it's a configuration issue that I can fix, but from what I've seen so far, it's just a pain in the rear.

OK... I've done some digging and my initial feelings about Tomcat were right. There are very specific directories that the JSP pages can go into. I can certainly understand the engine's need for this - it needs to generate files and then run them, but there's no technical reason that these files can't be located in any directory just like PHP and ASP files. I understand that most sites are static in their location, but once again, that seems to me to be a restriction that needn't apply. Ah... well... there's nothing I can do about it, and I'm certainly not impressed with the speed of JSP.