What's it all about, Alfie?

Late last night I got an email from Joel about problems he was having running CIA on the ISP's servers. So I called him this morning and tried to track down the problem. Well... there are a bunch of problems with this ISP. First, my account on the server is not put in the PostgreSQL database - and it needs to be so I can test the code. Also, I cannot see the contents of the /www/cell/webapp/ROOT/ directory where all the PHP files and the links to cia are maintained. Also, Joel cannot execute cia which is puzzeling because I've properly set up the permissions on the executables and even run them myself. But since I don't have access to the database, I can't test what Joel, or the web server, are trying to do. And since Joel can't execute the code he can't tell me what's wrong either.

So... the ISP needs to put me in the database as a user and also figure out why I can't see the /www/cell/webapp/ROOT/ directory. When those are cleared up I should be able to run the code and see what problems might be there. Then we can go about figuring out why Joel can't run the code. Lots to do, and it doesn't help that there seems to be so poor communication between the ISP and Joel, but that's life.

Another thing that happened this morning was really nice. I got an email back from Leslie, my cousin Mike's wife in VT. I had emailed them about getting some syrup because I've finally tired of the store-bought kind, and mentioned that if they were interested in setting up an eCommerce site for the farm, I'd be glad to help. And so she emailed back with their problems and I went about trying to track down solutions.

The first problem is getting a general eCommerce Shopping Site up. This is pretty easy these days for the Farm as their selection is fixed and doesn't occupy more than a page. I went to Freshmeat and did a serach on shopping systems. I got more than 40 - some Perl-based, others PHP... some used MySQL, others PostgreSQL, still others used files. In the end there certainly looks like there's more than enough to pick from. We need to know the capabilities of the ISP they're using to know which one(s) will work in their installation, so I've asked Leslie to check on it for me.

Her second concern was UPS shipping. When they ship meat - which they do a lot, they need it to arrive in a very timely fashion as it is not frozen. So if someone orders from Boston, it can go UPS Ground, but if the order is from California, then it needs to go Next Day. Thankfully, UPS has a series of free tools available on their eCommerce web site. They tend to work in concert with one another and can be used either by HTTP or by XML. UPS seems to have gone to a great deal of trouble to make these tools easily available to the eCommerce web site builders, and I am impressed. They have a tool that verifies the city, state and zip for correctness, and then another that calculates the shipping time for different shipping methods from the source to the destination. This is exactly what Leslie needs. She can get the city, state and zip from the web site, and then pass that to UPS for verification, and then get the shipping times and see what ones are really available to the customer. Then that can be integrated into the order and away we go.

I have to admit that I'm really interested in helping Mike out on this. I know it isn't much, but it's nice when we can reach out and help family. I mailed off all this info to Leslie and we'll see what she comes back with. I can build this all on tux or sparky and then when the time comes I can ship it to their ISP for deployment. It sounds pretty nice.

UPDATE: I exchanged a few more emails with Leslie and it looks like their ISP has an eCommerce option with database support and credit card processing. For $50/mo this might be their best option and I let them know about it. If I used their ISP and wanted me to put an eCommerce site on it, I'll have to stick with Perl/CGI and forgo any database. I mentioned this to her, and suggested that it might become necessary to move the site to another ISP that supports PHP/PostgreSQL/etc. for less than $50/mo.

This morning I decided to move tux from the ISDN network to the Cable Modem network. Since this was a move from static IP addressing to DHCP, I expected a few problems and planned for it to take me most of the day to get it right. I was very pleased to see that Linux had pump which handles the DHCP client tasks quite nicely. When I created an /etc/pump.conf with a set of very simple, reasonable defaults, things went along quite nicely. Within a few short minutes the NIC was reconfigured and running quite nicely on the DHCP address.

Then came the task of converting all the defaults and settings that went along with the old network. Not brain surgery, but a lot of little details - NTP, proxies, etc. all had to be reconfigured for the new network. At the same time I moved my Toshiba Portege 3015 to behind the NAT/firewall so that there would still only be four machines on the Cable Modem network. That was very simple as I maintained the DHCP configuration.

So inside of an hour I had reconfigured the networks to move tux to the outside world. You can jump to him here. Good tools make work easy.

This evening I spent some time working up the initial The Man from S.P.U.D. homepage. I spent a lot of time on the graphics, and I think it shows. The text is a little weak, but even that's not bad. I think it'll do nicely as a web page. I wanted to keep it simple and short. Nothing fancy, and easily viewed on all browsers. Nice work.

Today I decided to look at as many runtimes for CIA as I could reasonably afford to do. To start, we had the ISP vs. sparky, and that's a good place to start because I believe the ISP was running a dual Pentium III/600MHz machine so I don't need to even try to run CIA on anything else in my office. What I decided to do next was to check out the Compaq TestDrive systems where I could easily have temporary access to a lot of different hardware - most importantly to some fast Alpha machines to see if I can get the runtimes in the 30 sec. range.

Once you sign up, you get a simple shell account, and that's about it. It's enough for me as they are nice enough to install almost every package in RedHat 7 including PostgreSQL, GTK and all the development libraries. This means that the build on the ES40 took very little time, but I was surprised to see that the execution speed of the compiler wasn't faster than the ISP's dual 600 PIII... maybe it's true that Alpha's are excellent at floating point, but not anything great at general computation. We'll have to see.

The ES40 had a problem mounting the /house home directories, so the testing has been put on hold and I sent in an email about the problem. I'm all ready to do the testing when the mount point is re-established, and then compare this to the runs I did yesterday. I have no idea if it'll be any faster, but Compaq didn't have any faster PIII systems.

OK... they got the mount points working and I was able to run CIA on the ES40 Alpha system - what Compaq says is their highest powered machine in the Test Drive program. It seems right as it's got at least 8GB of RAM - probably more. Wow! Anyway, here are the numbers:

This shows that if we spend a bundle, it's possible to get the servicing times in the 30 sec. range. My guess is that the ES40 is at least $30,000 - but that's just a guess. It's one of their top alpha machines, and it shows. Nice times.

Late yesterday I decided to do a Google search on NeXTSTEP and SSH to see if anyone had built an SSH client for NeXTSTEP. What I found was the 'commercial' version of SSH that came in two versions - 1.x and 2.x. I got both and tried to get them both to compile. The 2.x version was hopeless due to signal differences, and the 1.x version needed me to write strdup() - but that wasn't bad. In the end I have a 1.x version of SSH which is a lot better than telnet.

Yesterday was a little slow because I didn't spend much time in the office. There were a lot of things that needing doing around the house, and so most of the day was spent away from this wonderful playroom. It's nice to be back this morning.

This evening I got word from Joel that the ISP guys had finally gotten around to installing GTK 1.2.8 development includes and libraries so I could get back to building CIA on their machines. Thankfully, it built fine, so now I'm in the process of transferring up a few slides so that I can test the speed and overall functionality. It should be fine, but I have no idea as to the speed issue. My guess is that it's reasonably fast, but nothing to write home about.

I also spent a bit of time today writing out my goals/plans/etc. for CIA and sent them to the rest of the guys. I'm hoping that this starts the process to either get serious about CIA or get out. I don't really care which, but if they wait too long I'll decide for them in what I do professionally.

UPDATE: I did a similar run on sparky and the ISP hosting the CIA website. I used the Dako #6 run with the following options: '-op -om -m' so as not to impact the FFTW time on one machine and not another. What I got was:

Which means that the ISP is running at about 28.2% of sparky. This is nice to know so that I can get a reasonable idea of production runtimes by looking at the runtimes on sparky and then multiplying by 0.282. Not like it's brain surgery, but it's nice to know.

Well... today is my last official day at BankOne and I have very mixed emotions. The package is nice, but I can't help wonder what would have happened if... Oh well... nothing I can do about that now.

I realized that I didn't have a consistent verison of PHP 4 on all three servers - sparky, barney and opus. Much to my surprise, sparky was the furthest from current running 4.0.3pl1. So I took the time to get opus to 4.0.4pl1 as well as building 4.0.4pl1 for sparky. I really like PHP and now that all the machines are at the current release we're in a good position to use either of the XML-RPC PHP clients. Nothing major, but since I had the time it seemed like a good thing to get caught up on.

I talked to Joel today - freshly back from vacation in the Florida Keys, and he suggested I get the CIA codebase up to the ISP and build it without PNG or FFTW support - as they have still not installed those components on the server. So I built a tar file and shipped it up with scp - gotta love those OpenSSH tools. Anyway, I get the code up and then go about modifying it for the above limitations. Not too bad, I think to myself.

Then I try to make it and get a ton of errors. Holy Cow! This server doesn't have the GTK includes installed on it! So I check and sure enough, the RPM gtk+-devel-1.2.8-7 is missing. So I send off a message to Joel to relay to them asking for this to be installed as well. I understand the desire to install 'less' rather than 'more' on servers where you're selling the free space, but the time they've lost in putting these on after the fact is more then offsetting the up-front lost revenue. Penny-wise, Pound-foolish.

When they get the GTK includes installed I'll be able to build CIA - with the limitations. If they get libpng installed, I can finish building Ploticus, and remove that limitation. Hmmm... service really is everything, isn't it?

UPDATE: - these guys at the ISP are amazing... I've been waiting to see when they get the GTK development package installed and have noticed that they only installed two of the four packages for VIM. Now it's true that they saved several MB of storage, but at what cost? Now gvim isn't there, nor is vim. The syntax highlighting is not there, but they have a form of VI... It's amazing.

They have a web-based technology look-up that does some kind of query to see what's installed on their machines, and it says gtk+-devel-1.2.8-7 was installed today, but the files aren't there, nor is the package listed in the RPM database for the machine I'm logged into. So they have a disconnect somewhere, and we're suffering for it. Oh well... what do you expect from a company that's doing this on the side... We might not have gotten a lot better service from a serious hosting site, but I have to wonder why it is that they are up and happy - with satisfied customers, and we're here fighting about what packages are installed. Holy Cow.

Today has been pretty uneventful. Late yesterday I had to re-arrange my 2kVA UPS as it was interferring with my SGI's monitor - a little wobble in the lower-right-hand corner of the display. Annoying, so I had to un-stack the units. Also, the stacked units were covering the CD-ROM drawer on tux so I needed to un-stack them regardless. At least now the distortion is acceptable, but I've got a lot of stuff in this office, and it's hard to keep the interference down.

I've run pupdate on sparky which updates my Solaris 7 installation with all the current patches from Sun. It's normally a real pain to do this by hand, but this script I wrote makes it a lot easier. It seems they keep cranking out patches or updates to existing patches on a regular basis and I need to keep checking to stay up to date.

I have received word from RedHat that there are a few packages that need updating and yet my traditional mirror doesn't have them yet. I've seen some of them on RedHat's ftp site, but I have always had problems getting things from RedHat's site simply because it's so busy. The files are there, and they'll make their way to the mirrors in a day or so, I just have to be patient.

Things are going well besides all this... it's quiet, but that's good too. I heard from another headhunter who saw my resume on a web site and wanted to present me to another company. I said "Sure", and we'll see if anything comes of it. No other word from the other places that I've been talking to. Tomorrow is my last official day at BankOne - after that I'm on the package which is both good and bad. Lots of emotions tied up into this place... of late, most of them are bad.

Well... Joesph's home from school, so that's good. I'll go play up there for a while and take the phone with me in case anyone calls. It's been a slow day for Science... wish I had Dexter's fountain of ideas to work from - that little guy's got a lot of spunk.

Late today I got a message from the project lead for bidwatcher asking me if I could go through the CVS files for bidwatcher and make sure everything is there. I jumped at the idea and made the changes and sent back a diff file for the latest changes. It's nice to be able to help out in projects like this. I really like doing it. Well... it looks like my changes will make it into the next release of bidwatcher. Yippee!

One of my current interests is the communication between machines for the purposes of solving a problem. Like having a 'calculation engine', or 'valuation server'... machines that did a non-GUI activity very fast and was available for anyone to use with a minimal of fuss. A few months ago I heard about XML-RPC and the more I read about it the more interested I became. I've always liked good, solid RPC designs - CORBA, RMI, but this is nice in that it has clients and servers for a host of languages - PHP, Perl, Tk, Python, C/C++, Java. These make it very easy to host on a platform that's just about anything you want, and have a client come in from just about anything.

This morning I decided to get into it a little more and picked up a PHP client/server implementation from Useful software that works fine on the PHP 4.03pl1 I have installed on sparky and barney. What I wanted to do was to set up a server on sparky and hit it from a client on barney.

The docs don't give you a lot of help here, but it doesn't take a lot to figure it out, either. Basically, you need to install the directory in a central location. I chose /usr/local/xmlrpc-php on sparky and /usr/xmlrpc-php on barney - these are reasonable defaults for Solaris and Irix, respectively. Then I edited the PHP files so the include() directives pointed to the installed directories. Finally, I linked the server.php file to my public_html directory on sparky and copied the client.php to my public_html directory on barney. I could then edit client to point to sparky and /~drbob/server.php.

At this point, I was able to go to barney and pull up http://barney/~drbob/client.php and it worked perfectly!. I couldn't have asked for a better result. The XML-RPC standard uses XML over HTTP, so it's a natural to have a server written in PHP. With all the capabilities PHP has, it's easy to see that any complex processing could be shelled out to any language and the only thing the PHP server needs to do is the basic interface. Very nice.

Now I'm in the neighborhood looking for a good project to use this on. One that comes to mind is the Quip Solver. It takes a cyphertext and a key and returns a plaintext. This would be OK, I suppose, but it's not exactly what I had in mind. Maybe I'll do it just for fun and practice... But I'm still looking for a better project.

Today I finally received the other half of my APC Smart-UPS 2000 that was supposed to arrive yesterday with the first half, but got lost on the truck and arrived today. After letting it sit for a few hours (as recommended by the docs), I moved it into place taking the 900VA UPS that was covering barney and putting it on wally. The 2kVA now covers tux and barney and is operating at a very low load. Excellent! It's nice to know that I can now put all three major machines on the net and have them protected nicely against outages.

Now I can see why people use NT - heck... I have two machines myself, and use them on a daily basis. But what's caused me trouble today is the DOS-Unix CRLF conversion problem. Specifically, The Weather Channel is now servicing two different pages/servers - one is Unix-based, and the other is NT-based. How do I know? One has redundant CR codes on every line and the other doesn't. Normally, I'd be OK with this, but sed doesn't allow for a regexp matching the control code. This makes it very hard to get rid of them. Thankfully, Solaris has dos2unix and it can strip the codes while leaving Unix files intact.

It just goes to show that once again, scraping screens is hard work when you can't pin down the screen to scrape. And while I understand load balancing, it really helps if the load balance between machines of the same type - at least of the same CRLF-design.

I also sent all my patches to the bidwatcher crew... I like the idea of having a single status line - as in the original version, but I understand the need for different UIs. Heck, we could even make it a preferences item and have them changed on the fly. Anyway, I sent in the patches and we'll see what they are going to do about it. I read on the forums that they are trying to get the 1.2.1 tar file updated, and maybe they'll incorporate these fixes and put up a 1.2.2. We'll have to see.

This morning I looked into two updates for some Open Source tools I use all the time - bidwatcher and GAIM. Now, previously I had said that I didn't like the stream of bugs in GAIM updates, but as long as I checked to see if the fixes I'd submitted are in, then I think it's a reasonable thing to try and compile it to see how it goes. Thankfully, it went well, and other than the problem of AOL blocking out the Oscar protocol from AIM-clones, it's fine.

I also tried to upgrade to bidwatcher 1.2.1 but the gzipped tar file on the site was improperly made and couldn't be gunzipped. Thankfully, the 1.2.0 file could be and I downloaded it and built it. What I saw was a very mixed bag, and as a result I'm staying with my modified 1.1.9.1 version.

The designers decided to update the code to make the right-clicking on the auctions easier, but then they added two status bars at the bottom of the window. Now I don't claim to be a UI expert, but that's almost by definition excessive. They have one for error messages and the other for bid status updates. Now if you have an error, then the bid status has got to be bad, and if you don't then you have no need to display an error, and the status is seen. Therefore, there's only reason to have one. One - that's all.

So I sent off a few messages to the bidwater forums and we'll see what happens. I think I'll patch the code to have a single status bar just because I can't imagine what on earth they must have been thinking of. The best solution is to use the status bar at the top of the window for all needs and leave it at that. Maybe that's what I'll put in and see how it flies.

I also need to make some calls about the interviews and call about the new 2kVA UPS for the office. Not a lot, but enough to keep me off the streets as they say...

UPDATE: I fixed the 1.2.0 version of bidwatcher to use a single status line - the one that already existed in the older version, but also allowed the new users to have their dual-status bar UI. I've emailed the maintainers to see if they're interested in the optional improvements. We'll see if they come back interested.

I called about the interviews and told the headhunters how it went. I also found out that one of the places I was really interested in was getting back in touch with them. It might be really nice to hear from a few of these places this week. Good.

Finally, I called about the UPS and it should be here tomorrow - they say. What I'm hoping for is to be able to switch out one of my 900VA for this new 2kVA and put barney and one of tux's supplies on it. I figure that's about 1000W which should give me about 23 min of support on the 2kVA. At 115 lbs. this is a serious UPS which is very much in need in this office. Anyway... then I'll take the 900VA and put it on wally the NeXTSTEP machine and get rid of that awful TrippLite that I never should have gotten in the first place.

Today in a slightly different order...

I had some really good interviews on Friday. Glad to have them. Both groups thought I was more than qualified for their projects/positions, and I was able to really talk to both of them at the depth that I want to to try and get to know someone. The first person believed they had the ideal group I've always been looking for - but I think he was a little too impressed with my qualifications. I got the sense after he said "You're more than qualified for any position we have..." that he was concerned that I might not like being at the level I would be at in his group. Don't know. I'll certainly have to learn more about that place before I can make a decision.

The second place was a gas - huge new project, lots of problems, lots of visibility, very little time. RedHat Linux is a target platform, and that's an overall combination that's very attractive to me. The work would be in a large distributed computing environment and there would be all kinds of challenges to overcome. When we pulled it off, it would certainly be an impressive feat. Very much a career highlight project.

Don't know what will happen, but it's great to have good interviews.

I had a problem on Saturday with getting the weather forecast from The Weather Channel. I think the most likely possibility is that the MediaOne cable modem went down. This is aggrivating. What I decided I needed to do was to make it a little more robust and put in a looping mechanism that would re-try the grab after a delay of 10 mins. This seemed reasonable, and I put in a fail-safe that if the time became 11:00 am, it'd fail outright. Hopefully, this will mean that I have less problems missing data from the moring pulls.

On the CIA front, Joel has been trying to get a lot of work done as quickly as possible in the last few days of this week. He's going on vacation next week, and wanted to get something for Dako to look at before he left. I had plans, and that made for a conflict that I'm not at all sure Joel was happy with the outcome. But that's life... The Dako slides seem to be giving the code a hard time, but I honestly haven't done more than put the slides in the system, and run two of them through. I didn't spend any time looking at the results, but when I do we'll see how well we're handling the Her2 process, in general.

Also, it's the end of March, and it's snowing outside. I'm tired of the cold. This has been a very long, hard winter, and I'm ready for a break.