What's it all about, Alfie?

Well... I've been working hard over the weekend to get the web site for my cousin cleaned up and updated. It's not yet in production, so no link, but soon it will be and then I'll include one. I've added lots of things - a FAQ that Leslie typed up, and changed the verbage on several pages to be more what she wanted. I've also finally received word from the ISP hosters of their site as to what tools they have that I can take advantage of. Interestingly enough, it's not a really great place to host.

For example, if we wanted to use SSL to get confidential information, then we'd have to get a Certificate from Verisign for $349/yr and then pay the ISP another $120/yr to open up SSL on the box. This, in addition to a $100 set-up fee means that security is a big jump in cost to the site. And frankly I'm not sure it's really needed. What they have now is nothing like what a full-blown eCommerce site is, and I'm not sure how much an eCommerce site is going to boost their sales.

To get a serious site going, it's going to take a lot more money, and that's up to them. So I've explained it to them and we'll see what they think. For now, the site is lookg very nice in development and we're getting close to a move to production.

I've also signed up with easyDNS for the domain TheManFromSPUD.com - something I've really wanted to do for a while. It was very easy, and had it not been for the problems with payment, it would have been exceptionally easy. It looks very easy to use and in a few days I'll be able to move all my machines to this new domain and that will be a real treat.

Late last night I got an email from Joel about problems he was having running CIA on the ISP's servers. So I called him this morning and tried to track down the problem. Well... there are a bunch of problems with this ISP. First, my account on the server is not put in the PostgreSQL database - and it needs to be so I can test the code. Also, I cannot see the contents of the /www/cell/webapp/ROOT/ directory where all the PHP files and the links to cia are maintained. Also, Joel cannot execute cia which is puzzeling because I've properly set up the permissions on the executables and even run them myself. But since I don't have access to the database, I can't test what Joel, or the web server, are trying to do. And since Joel can't execute the code he can't tell me what's wrong either.

So... the ISP needs to put me in the database as a user and also figure out why I can't see the /www/cell/webapp/ROOT/ directory. When those are cleared up I should be able to run the code and see what problems might be there. Then we can go about figuring out why Joel can't run the code. Lots to do, and it doesn't help that there seems to be so poor communication between the ISP and Joel, but that's life.

Another thing that happened this morning was really nice. I got an email back from Leslie, my cousin Mike's wife in VT. I had emailed them about getting some syrup because I've finally tired of the store-bought kind, and mentioned that if they were interested in setting up an eCommerce site for the farm, I'd be glad to help. And so she emailed back with their problems and I went about trying to track down solutions.

The first problem is getting a general eCommerce Shopping Site up. This is pretty easy these days for the Farm as their selection is fixed and doesn't occupy more than a page. I went to Freshmeat and did a serach on shopping systems. I got more than 40 - some Perl-based, others PHP... some used MySQL, others PostgreSQL, still others used files. In the end there certainly looks like there's more than enough to pick from. We need to know the capabilities of the ISP they're using to know which one(s) will work in their installation, so I've asked Leslie to check on it for me.

Her second concern was UPS shipping. When they ship meat - which they do a lot, they need it to arrive in a very timely fashion as it is not frozen. So if someone orders from Boston, it can go UPS Ground, but if the order is from California, then it needs to go Next Day. Thankfully, UPS has a series of free tools available on their eCommerce web site. They tend to work in concert with one another and can be used either by HTTP or by XML. UPS seems to have gone to a great deal of trouble to make these tools easily available to the eCommerce web site builders, and I am impressed. They have a tool that verifies the city, state and zip for correctness, and then another that calculates the shipping time for different shipping methods from the source to the destination. This is exactly what Leslie needs. She can get the city, state and zip from the web site, and then pass that to UPS for verification, and then get the shipping times and see what ones are really available to the customer. Then that can be integrated into the order and away we go.

I have to admit that I'm really interested in helping Mike out on this. I know it isn't much, but it's nice when we can reach out and help family. I mailed off all this info to Leslie and we'll see what she comes back with. I can build this all on tux or sparky and then when the time comes I can ship it to their ISP for deployment. It sounds pretty nice.

UPDATE: I exchanged a few more emails with Leslie and it looks like their ISP has an eCommerce option with database support and credit card processing. For $50/mo this might be their best option and I let them know about it. If I used their ISP and wanted me to put an eCommerce site on it, I'll have to stick with Perl/CGI and forgo any database. I mentioned this to her, and suggested that it might become necessary to move the site to another ISP that supports PHP/PostgreSQL/etc. for less than $50/mo.

Late yesterday I decided to do a Google search on NeXTSTEP and SSH to see if anyone had built an SSH client for NeXTSTEP. What I found was the 'commercial' version of SSH that came in two versions - 1.x and 2.x. I got both and tried to get them both to compile. The 2.x version was hopeless due to signal differences, and the 1.x version needed me to write strdup() - but that wasn't bad. In the end I have a 1.x version of SSH which is a lot better than telnet.

Yesterday was a little slow because I didn't spend much time in the office. There were a lot of things that needing doing around the house, and so most of the day was spent away from this wonderful playroom. It's nice to be back this morning.

This evening I got word from Joel that the ISP guys had finally gotten around to installing GTK 1.2.8 development includes and libraries so I could get back to building CIA on their machines. Thankfully, it built fine, so now I'm in the process of transferring up a few slides so that I can test the speed and overall functionality. It should be fine, but I have no idea as to the speed issue. My guess is that it's reasonably fast, but nothing to write home about.

I also spent a bit of time today writing out my goals/plans/etc. for CIA and sent them to the rest of the guys. I'm hoping that this starts the process to either get serious about CIA or get out. I don't really care which, but if they wait too long I'll decide for them in what I do professionally.

UPDATE: I did a similar run on sparky and the ISP hosting the CIA website. I used the Dako #6 run with the following options: '-op -om -m' so as not to impact the FFTW time on one machine and not another. What I got was:

Which means that the ISP is running at about 28.2% of sparky. This is nice to know so that I can get a reasonable idea of production runtimes by looking at the runtimes on sparky and then multiplying by 0.282. Not like it's brain surgery, but it's nice to know.

Well... today is my last official day at BankOne and I have very mixed emotions. The package is nice, but I can't help wonder what would have happened if... Oh well... nothing I can do about that now.

I realized that I didn't have a consistent verison of PHP 4 on all three servers - sparky, barney and opus. Much to my surprise, sparky was the furthest from current running 4.0.3pl1. So I took the time to get opus to 4.0.4pl1 as well as building 4.0.4pl1 for sparky. I really like PHP and now that all the machines are at the current release we're in a good position to use either of the XML-RPC PHP clients. Nothing major, but since I had the time it seemed like a good thing to get caught up on.

I talked to Joel today - freshly back from vacation in the Florida Keys, and he suggested I get the CIA codebase up to the ISP and build it without PNG or FFTW support - as they have still not installed those components on the server. So I built a tar file and shipped it up with scp - gotta love those OpenSSH tools. Anyway, I get the code up and then go about modifying it for the above limitations. Not too bad, I think to myself.

Then I try to make it and get a ton of errors. Holy Cow! This server doesn't have the GTK includes installed on it! So I check and sure enough, the RPM gtk+-devel-1.2.8-7 is missing. So I send off a message to Joel to relay to them asking for this to be installed as well. I understand the desire to install 'less' rather than 'more' on servers where you're selling the free space, but the time they've lost in putting these on after the fact is more then offsetting the up-front lost revenue. Penny-wise, Pound-foolish.

When they get the GTK includes installed I'll be able to build CIA - with the limitations. If they get libpng installed, I can finish building Ploticus, and remove that limitation. Hmmm... service really is everything, isn't it?

UPDATE: - these guys at the ISP are amazing... I've been waiting to see when they get the GTK development package installed and have noticed that they only installed two of the four packages for VIM. Now it's true that they saved several MB of storage, but at what cost? Now gvim isn't there, nor is vim. The syntax highlighting is not there, but they have a form of VI... It's amazing.

They have a web-based technology look-up that does some kind of query to see what's installed on their machines, and it says gtk+-devel-1.2.8-7 was installed today, but the files aren't there, nor is the package listed in the RPM database for the machine I'm logged into. So they have a disconnect somewhere, and we're suffering for it. Oh well... what do you expect from a company that's doing this on the side... We might not have gotten a lot better service from a serious hosting site, but I have to wonder why it is that they are up and happy - with satisfied customers, and we're here fighting about what packages are installed. Holy Cow.

One of my current interests is the communication between machines for the purposes of solving a problem. Like having a 'calculation engine', or 'valuation server'... machines that did a non-GUI activity very fast and was available for anyone to use with a minimal of fuss. A few months ago I heard about XML-RPC and the more I read about it the more interested I became. I've always liked good, solid RPC designs - CORBA, RMI, but this is nice in that it has clients and servers for a host of languages - PHP, Perl, Tk, Python, C/C++, Java. These make it very easy to host on a platform that's just about anything you want, and have a client come in from just about anything.

This morning I decided to get into it a little more and picked up a PHP client/server implementation from Useful software that works fine on the PHP 4.03pl1 I have installed on sparky and barney. What I wanted to do was to set up a server on sparky and hit it from a client on barney.

The docs don't give you a lot of help here, but it doesn't take a lot to figure it out, either. Basically, you need to install the directory in a central location. I chose /usr/local/xmlrpc-php on sparky and /usr/xmlrpc-php on barney - these are reasonable defaults for Solaris and Irix, respectively. Then I edited the PHP files so the include() directives pointed to the installed directories. Finally, I linked the server.php file to my public_html directory on sparky and copied the client.php to my public_html directory on barney. I could then edit client to point to sparky and /~drbob/server.php.

At this point, I was able to go to barney and pull up http://barney/~drbob/client.php and it worked perfectly!. I couldn't have asked for a better result. The XML-RPC standard uses XML over HTTP, so it's a natural to have a server written in PHP. With all the capabilities PHP has, it's easy to see that any complex processing could be shelled out to any language and the only thing the PHP server needs to do is the basic interface. Very nice.

Now I'm in the neighborhood looking for a good project to use this on. One that comes to mind is the Quip Solver. It takes a cyphertext and a key and returns a plaintext. This would be OK, I suppose, but it's not exactly what I had in mind. Maybe I'll do it just for fun and practice... But I'm still looking for a better project.

Today I finally received the other half of my APC Smart-UPS 2000 that was supposed to arrive yesterday with the first half, but got lost on the truck and arrived today. After letting it sit for a few hours (as recommended by the docs), I moved it into place taking the 900VA UPS that was covering barney and putting it on wally. The 2kVA now covers tux and barney and is operating at a very low load. Excellent! It's nice to know that I can now put all three major machines on the net and have them protected nicely against outages.

Now I can see why people use NT - heck... I have two machines myself, and use them on a daily basis. But what's caused me trouble today is the DOS-Unix CRLF conversion problem. Specifically, The Weather Channel is now servicing two different pages/servers - one is Unix-based, and the other is NT-based. How do I know? One has redundant CR codes on every line and the other doesn't. Normally, I'd be OK with this, but sed doesn't allow for a regexp matching the control code. This makes it very hard to get rid of them. Thankfully, Solaris has dos2unix and it can strip the codes while leaving Unix files intact.

It just goes to show that once again, scraping screens is hard work when you can't pin down the screen to scrape. And while I understand load balancing, it really helps if the load balance between machines of the same type - at least of the same CRLF-design.

I also sent all my patches to the bidwatcher crew... I like the idea of having a single status line - as in the original version, but I understand the need for different UIs. Heck, we could even make it a preferences item and have them changed on the fly. Anyway, I sent in the patches and we'll see what they are going to do about it. I read on the forums that they are trying to get the 1.2.1 tar file updated, and maybe they'll incorporate these fixes and put up a 1.2.2. We'll have to see.

This morning I looked into two updates for some Open Source tools I use all the time - bidwatcher and GAIM. Now, previously I had said that I didn't like the stream of bugs in GAIM updates, but as long as I checked to see if the fixes I'd submitted are in, then I think it's a reasonable thing to try and compile it to see how it goes. Thankfully, it went well, and other than the problem of AOL blocking out the Oscar protocol from AIM-clones, it's fine.

I also tried to upgrade to bidwatcher 1.2.1 but the gzipped tar file on the site was improperly made and couldn't be gunzipped. Thankfully, the 1.2.0 file could be and I downloaded it and built it. What I saw was a very mixed bag, and as a result I'm staying with my modified 1.1.9.1 version.

The designers decided to update the code to make the right-clicking on the auctions easier, but then they added two status bars at the bottom of the window. Now I don't claim to be a UI expert, but that's almost by definition excessive. They have one for error messages and the other for bid status updates. Now if you have an error, then the bid status has got to be bad, and if you don't then you have no need to display an error, and the status is seen. Therefore, there's only reason to have one. One - that's all.

So I sent off a few messages to the bidwater forums and we'll see what happens. I think I'll patch the code to have a single status bar just because I can't imagine what on earth they must have been thinking of. The best solution is to use the status bar at the top of the window for all needs and leave it at that. Maybe that's what I'll put in and see how it flies.

I also need to make some calls about the interviews and call about the new 2kVA UPS for the office. Not a lot, but enough to keep me off the streets as they say...

UPDATE: I fixed the 1.2.0 version of bidwatcher to use a single status line - the one that already existed in the older version, but also allowed the new users to have their dual-status bar UI. I've emailed the maintainers to see if they're interested in the optional improvements. We'll see if they come back interested.

I called about the interviews and told the headhunters how it went. I also found out that one of the places I was really interested in was getting back in touch with them. It might be really nice to hear from a few of these places this week. Good.

Finally, I called about the UPS and it should be here tomorrow - they say. What I'm hoping for is to be able to switch out one of my 900VA for this new 2kVA and put barney and one of tux's supplies on it. I figure that's about 1000W which should give me about 23 min of support on the 2kVA. At 115 lbs. this is a serious UPS which is very much in need in this office. Anyway... then I'll take the 900VA and put it on wally the NeXTSTEP machine and get rid of that awful TrippLite that I never should have gotten in the first place.

Today in a slightly different order...

I had some really good interviews on Friday. Glad to have them. Both groups thought I was more than qualified for their projects/positions, and I was able to really talk to both of them at the depth that I want to to try and get to know someone. The first person believed they had the ideal group I've always been looking for - but I think he was a little too impressed with my qualifications. I got the sense after he said "You're more than qualified for any position we have..." that he was concerned that I might not like being at the level I would be at in his group. Don't know. I'll certainly have to learn more about that place before I can make a decision.

The second place was a gas - huge new project, lots of problems, lots of visibility, very little time. RedHat Linux is a target platform, and that's an overall combination that's very attractive to me. The work would be in a large distributed computing environment and there would be all kinds of challenges to overcome. When we pulled it off, it would certainly be an impressive feat. Very much a career highlight project.

Don't know what will happen, but it's great to have good interviews.

I had a problem on Saturday with getting the weather forecast from The Weather Channel. I think the most likely possibility is that the MediaOne cable modem went down. This is aggrivating. What I decided I needed to do was to make it a little more robust and put in a looping mechanism that would re-try the grab after a delay of 10 mins. This seemed reasonable, and I put in a fail-safe that if the time became 11:00 am, it'd fail outright. Hopefully, this will mean that I have less problems missing data from the moring pulls.

On the CIA front, Joel has been trying to get a lot of work done as quickly as possible in the last few days of this week. He's going on vacation next week, and wanted to get something for Dako to look at before he left. I had plans, and that made for a conflict that I'm not at all sure Joel was happy with the outcome. But that's life... The Dako slides seem to be giving the code a hard time, but I honestly haven't done more than put the slides in the system, and run two of them through. I didn't spend any time looking at the results, but when I do we'll see how well we're handling the Her2 process, in general.

Also, it's the end of March, and it's snowing outside. I'm tired of the cold. This has been a very long, hard winter, and I'm ready for a break.

I had to change the script that processes the page from The Weather Channel's site because they changed the format of the information. Not a big deal, but when you're scraping stuff off the pages, it's a constant battle to stay abreast with their changes. It would be much nicer if they simply offered a database link that I could connect to... 🙂 In any case, the change was pretty easy to make once I knew what the problem was.

Then I received a call back from one of the headhunters I'd been dealing with in my latest round of job searches. The company I had interviewed with a few days ago was still uncertain about me - and the only negative things they could think of were the fact that I didn't wear a suit, and what they considered to be my limited experience with C++.

Now I don't usually rant in this journal, but here's something I'm going to rant about... I've been doing C++ work for over 15 years - OK, not every day all 15 years, but I don't write Java every day, all day, and I'm still outstanding at it. Yes, I've taken a few years off from C++ - sticking primarily to C and Java and web tools, but I'm as good in C++ as I ever was. More importantly, the technical interviews I've had with this company have all seemed quite fine with my technical skills. So much so, that they've said to my face the C++ isn't an issue. But to the headhunter they say different.

So it's one of two things - they are looking for a reason to ding me, or they aren't honest to me or the headhunter. Either is bad. Both means I don't like them.

Then there's the dress code. Now I'm not a slob - I'm very neat in appearance, and take great pride in a clean cut appearance. I bathe regularly, brush my teeth twice a day, and make sure to wear clean, pressed, clothes.

But not a suit.

I don't mind if others want to wear one, and I'll admit that there may be deep-rooted emotional damage in my background that is the root cause for this aversion to suits, but I just won't wear them. I don't own one. I can't image buying one. I own three ties, and only wear them and a sports coat in times of death and marriage. I look very nice in a button-down collar Lands End shirt and slacks. I always have polished shoes, and never try to pass off "hiking boots" as business shoes.

But that wasn't good enough for this one person. No... he was surprised that I didn't wear a suit to meet with the President of the company. Now mind you, the President himself wasn't surprised, it's this other person... but that's beside the point. The person making this judgement upon me was unkempt, wore hiking boots, and for an IT Director had the most incredibly messy desk I've ever seen. While a certain amount of clutter is normal, once the clutter stands for more than a few weeks, it fails to have any value and just indicates sloppy organizational skills. This was the man passing judgement on my dress...

Thankfully, experiences like this make me all the more convinced that this is not a place I want to work. For if they make these judgements in light of their own incredibly poor showing, then it's a situation that's bound to be bad sooner or later. I'm a very neat and orderly person, and I couldn't care a whit about what someone wears - so long as it's neat and clean. But if this place, or even just this person, is making these kinds of judgements about me, and then not being honest to either me or the headhunter, then I'm glad they're only so-so about me. I'm certainly only so-so about them.

I'm willing to give everyone a second-chance. I think a lot of the problems we have these days is because people rush to judgement and don't give second-chances. So if this place were to call me and talk up-font about these issues, I'd probably be fine with it. After all, maybe he'd had a bad day and maybe he worshiped the ground the President walked on. I don't know. But I do know that one rush to judgement is fine - so long as it's talked out. Failing to talk it out, or making another is unacceptable for me.

And so the search continues...

Today I decided to clean up the Her2 processing of CIA. Basically, we needed to have a controlled two-pass system - the first to gather the stats on the cell nuclie and the second to gather the stats on the stained membranes. This needed to be thought out before coding as the file names, as well as database records, would tend to walk all over each other. With a little help from this journal's design I decided to be smart about it and have a single command-line argument (-m) indicate to CIA that it needed to do both passes.

First, I had to differentiate which pass I was on - that was pretty easy, actually. Then when I had that, I just modified the names of the output files to include a "_nuc" in the middle of the name for the first, 'nuclear', pass. This worked out great. Then I made the main code fork off a second process (could have used multiprocessors here but opted against that) and have it do the first pass, then when it finished, the second pass on the stained matter would work. In truth, there's no conflict between the passes, and it could have easily been done at the same time. It's an easy change, and if we get the processor horsepower I can do that. But for now, it's serial.

When I ran this it all worked beautifully. I was really happy that I could easily add in the default options when dealing with a membrane sample - like ditch the aspect ratio, etc. Very nice indeed. So now it has automatically generated filters on both passes, and everything is ready for production data. I'm pretty happy with the clean-up.

I'm still unable to build this on the production site, but that's up to Joel as much as anything - they don't seem to be as responsive to me as to him, so I'll just send on what I need them to do to him and let him deal with it. I was trying to get Ploticus to compile, but needed the XFree86-devel RPM loaded, and so I sent a message off to him. After Ploticus, then we need FFTW and then we can build CIA.