What's it all about, Alfie?

Yesterday I received yet another notice about an ISP that allowed MySQL/PHP/Apache services, but not PostgreSQL. I wanted to find out what I didn't know about MySQL so I started looking into it on barney which has it installed.

Well... I certainly know why they like it. The entire database is file-based. This makes for easy accounting - doesn't it. Since an ISP wants to limit the entire diskspace per client, it makes sense that a database that's file-based is a good thing. Well... the fact is, PostgreSQL is also file-based, but the ownership of the files is that of the process running the database. Is it possible to still calculate the total used space? Yes. But it's more difficult, that's true. So now it's clear why. Let's see what it costs us.

PostgreSQL is very much like any of the high-end commercial packages. Here are a few things that MySQL doesn't have that PostgreSQL does:

• Foreign Keys - this is almost hard to believe. They even say in their documentation that this is a good thing as FKs are difficult to deal with. Now I'm no database snob, but I can't believe that anyone who maintains business critical data thinks this. FKs are the key to referential integrity and without that you have no guarantees that the data you have is related to the key components for it's understanding. Imagine an order item with $1,000 without an order to go with it? Do you assume it's bad data, or do you try and find the customer that's missing a $1,000 item, or find where your receivables are? This is the kind of thinking that I simply can't believe they didn't do.
• Stored Procedures - if you can parse SQL - which they do, then you should be able to parse scripts. If you can do that you can handle stored procedures. There's nothing to it. While it may not be the most efficient language to use, a few additions to straight SQL make it very reasonable for SPs. I'm really amazed that they don't consider this to be a useful addition. But looking at their attitude towards FKs, this doesn't surprise me.
• Transactions - for a database to be SQL-based, I'm amazed that such a fundamental concept can be thoughtfully left out. While I can certainly understand that there are plenty of applications that don't need transactions, there really aren't many that are business critical that don't. This then places the target market for MySQL squarely at the web server, text logs, non-business critical database market. There really isn't any other way to look at all these omissions other than to realize that there is a different market for this database.
• Triggers - this is a little easier to believe because a lot has to go into handling triggers. You ought to have a transaction to have a trigger so you can roll it back if the trigger senses that something's gone wrong.

Combine this with the fact that PostgreSQL is every bit as fast as MySQL and you're in the position of choosing a solid, complete database versus a table/record database. I can't even image why someone you willingly choose MySQL. I can understand why certain shops don't offer PostgreSQL, but that may be as much education as anything else. We'll have to see.

I got word from the ISP for North Hollow Farm that the account was set up for the shopping cart experience. I went and checked it out. I also checked into the UPS tools like address verification, time tracking and package pricing and services. There is good news and bad news about each, I'm afraid.

Most importantly, the shopping cart is reasonably well done with just enough customization to be nice, while still making it wasy for a new person to use. This will be important because there's a lot of data to load/type in, and while I can physically do it, she might as well do it because she knows it, and the maintenance practice will do her good. The bad news is that the order flow is fixed, and there's no way to interject the UPS tools into the flow if they aren't done by the software manufacturer. This isn't really surprising to me, but it's unfortunate.

On the UPS software, even if we could have interrupted the order flow, there's a problem with the UPS tools that I hadn't counted on - XML. The documentation said that the tools were available in HTML and XML forms, but that's only true for the most simplistic of tools. If we needed to address verification or time tracking, then we'd have to have XML, and that means CGI at least, and that's a server configuration issue. Not a pretty sight, I promise you. So... it's probably just as well that the order flow can't be modified - I'm not sure we could have modified it anyway.

This brings me to an idea... I wonder if I should just do it myself and then see how it compares. The issue really isn't anything other than what's best for Leslie. I can't really do card processing, but I can get an interface and try... I haven't done SSL/HTTPS, but I can find out about it and try... I guess it comes down to having the time to try versus doing something else.

I think I'll wait and see what Leslie says. Depending on me for code is not necessarily in her best interests.

I was doing a little reading yesterday and have to say that I'm more than a little disappointed in the state of JSP engines like Tomcat. If you look at similar scripting engines like PHP or even Microsoft's ASP, the web server is smart enough to know what to do based on the file extansion. But not Tomcat. It requires not only the file extension, but also the location. This is what I find a real pain.

I'm going to look into it more today to see what I can accomplish. Maybe it's a configuration issue that I can fix, but from what I've seen so far, it's just a pain in the rear.

OK... I've done some digging and my initial feelings about Tomcat were right. There are very specific directories that the JSP pages can go into. I can certainly understand the engine's need for this - it needs to generate files and then run them, but there's no technical reason that these files can't be located in any directory just like PHP and ASP files. I understand that most sites are static in their location, but once again, that seems to me to be a restriction that needn't apply. Ah... well... there's nothing I can do about it, and I'm certainly not impressed with the speed of JSP.

Well... I've been working hard over the weekend to get the web site for my cousin cleaned up and updated. It's not yet in production, so no link, but soon it will be and then I'll include one. I've added lots of things - a FAQ that Leslie typed up, and changed the verbage on several pages to be more what she wanted. I've also finally received word from the ISP hosters of their site as to what tools they have that I can take advantage of. Interestingly enough, it's not a really great place to host.

For example, if we wanted to use SSL to get confidential information, then we'd have to get a Certificate from Verisign for $349/yr and then pay the ISP another $120/yr to open up SSL on the box. This, in addition to a $100 set-up fee means that security is a big jump in cost to the site. And frankly I'm not sure it's really needed. What they have now is nothing like what a full-blown eCommerce site is, and I'm not sure how much an eCommerce site is going to boost their sales.

To get a serious site going, it's going to take a lot more money, and that's up to them. So I've explained it to them and we'll see what they think. For now, the site is lookg very nice in development and we're getting close to a move to production.

I've also signed up with easyDNS for the domain TheManFromSPUD.com - something I've really wanted to do for a while. It was very easy, and had it not been for the problems with payment, it would have been exceptionally easy. It looks very easy to use and in a few days I'll be able to move all my machines to this new domain and that will be a real treat.

Late last night I got an email from Joel about problems he was having running CIA on the ISP's servers. So I called him this morning and tried to track down the problem. Well... there are a bunch of problems with this ISP. First, my account on the server is not put in the PostgreSQL database - and it needs to be so I can test the code. Also, I cannot see the contents of the /www/cell/webapp/ROOT/ directory where all the PHP files and the links to cia are maintained. Also, Joel cannot execute cia which is puzzeling because I've properly set up the permissions on the executables and even run them myself. But since I don't have access to the database, I can't test what Joel, or the web server, are trying to do. And since Joel can't execute the code he can't tell me what's wrong either.

So... the ISP needs to put me in the database as a user and also figure out why I can't see the /www/cell/webapp/ROOT/ directory. When those are cleared up I should be able to run the code and see what problems might be there. Then we can go about figuring out why Joel can't run the code. Lots to do, and it doesn't help that there seems to be so poor communication between the ISP and Joel, but that's life.

Another thing that happened this morning was really nice. I got an email back from Leslie, my cousin Mike's wife in VT. I had emailed them about getting some syrup because I've finally tired of the store-bought kind, and mentioned that if they were interested in setting up an eCommerce site for the farm, I'd be glad to help. And so she emailed back with their problems and I went about trying to track down solutions.

The first problem is getting a general eCommerce Shopping Site up. This is pretty easy these days for the Farm as their selection is fixed and doesn't occupy more than a page. I went to Freshmeat and did a serach on shopping systems. I got more than 40 - some Perl-based, others PHP... some used MySQL, others PostgreSQL, still others used files. In the end there certainly looks like there's more than enough to pick from. We need to know the capabilities of the ISP they're using to know which one(s) will work in their installation, so I've asked Leslie to check on it for me.

Her second concern was UPS shipping. When they ship meat - which they do a lot, they need it to arrive in a very timely fashion as it is not frozen. So if someone orders from Boston, it can go UPS Ground, but if the order is from California, then it needs to go Next Day. Thankfully, UPS has a series of free tools available on their eCommerce web site. They tend to work in concert with one another and can be used either by HTTP or by XML. UPS seems to have gone to a great deal of trouble to make these tools easily available to the eCommerce web site builders, and I am impressed. They have a tool that verifies the city, state and zip for correctness, and then another that calculates the shipping time for different shipping methods from the source to the destination. This is exactly what Leslie needs. She can get the city, state and zip from the web site, and then pass that to UPS for verification, and then get the shipping times and see what ones are really available to the customer. Then that can be integrated into the order and away we go.

I have to admit that I'm really interested in helping Mike out on this. I know it isn't much, but it's nice when we can reach out and help family. I mailed off all this info to Leslie and we'll see what she comes back with. I can build this all on tux or sparky and then when the time comes I can ship it to their ISP for deployment. It sounds pretty nice.

UPDATE: I exchanged a few more emails with Leslie and it looks like their ISP has an eCommerce option with database support and credit card processing. For $50/mo this might be their best option and I let them know about it. If I used their ISP and wanted me to put an eCommerce site on it, I'll have to stick with Perl/CGI and forgo any database. I mentioned this to her, and suggested that it might become necessary to move the site to another ISP that supports PHP/PostgreSQL/etc. for less than $50/mo.

This morning I decided to move tux from the ISDN network to the Cable Modem network. Since this was a move from static IP addressing to DHCP, I expected a few problems and planned for it to take me most of the day to get it right. I was very pleased to see that Linux had pump which handles the DHCP client tasks quite nicely. When I created an /etc/pump.conf with a set of very simple, reasonable defaults, things went along quite nicely. Within a few short minutes the NIC was reconfigured and running quite nicely on the DHCP address.

Then came the task of converting all the defaults and settings that went along with the old network. Not brain surgery, but a lot of little details - NTP, proxies, etc. all had to be reconfigured for the new network. At the same time I moved my Toshiba Portege 3015 to behind the NAT/firewall so that there would still only be four machines on the Cable Modem network. That was very simple as I maintained the DHCP configuration.

So inside of an hour I had reconfigured the networks to move tux to the outside world. You can jump to him here. Good tools make work easy.

This evening I spent some time working up the initial The Man from S.P.U.D. homepage. I spent a lot of time on the graphics, and I think it shows. The text is a little weak, but even that's not bad. I think it'll do nicely as a web page. I wanted to keep it simple and short. Nothing fancy, and easily viewed on all browsers. Nice work.

Today I decided to look at as many runtimes for CIA as I could reasonably afford to do. To start, we had the ISP vs. sparky, and that's a good place to start because I believe the ISP was running a dual Pentium III/600MHz machine so I don't need to even try to run CIA on anything else in my office. What I decided to do next was to check out the Compaq TestDrive systems where I could easily have temporary access to a lot of different hardware - most importantly to some fast Alpha machines to see if I can get the runtimes in the 30 sec. range.

Once you sign up, you get a simple shell account, and that's about it. It's enough for me as they are nice enough to install almost every package in RedHat 7 including PostgreSQL, GTK and all the development libraries. This means that the build on the ES40 took very little time, but I was surprised to see that the execution speed of the compiler wasn't faster than the ISP's dual 600 PIII... maybe it's true that Alpha's are excellent at floating point, but not anything great at general computation. We'll have to see.

The ES40 had a problem mounting the /house home directories, so the testing has been put on hold and I sent in an email about the problem. I'm all ready to do the testing when the mount point is re-established, and then compare this to the runs I did yesterday. I have no idea if it'll be any faster, but Compaq didn't have any faster PIII systems.

OK... they got the mount points working and I was able to run CIA on the ES40 Alpha system - what Compaq says is their highest powered machine in the Test Drive program. It seems right as it's got at least 8GB of RAM - probably more. Wow! Anyway, here are the numbers:

This shows that if we spend a bundle, it's possible to get the servicing times in the 30 sec. range. My guess is that the ES40 is at least $30,000 - but that's just a guess. It's one of their top alpha machines, and it shows. Nice times.

Late yesterday I decided to do a Google search on NeXTSTEP and SSH to see if anyone had built an SSH client for NeXTSTEP. What I found was the 'commercial' version of SSH that came in two versions - 1.x and 2.x. I got both and tried to get them both to compile. The 2.x version was hopeless due to signal differences, and the 1.x version needed me to write strdup() - but that wasn't bad. In the end I have a 1.x version of SSH which is a lot better than telnet.

Yesterday was a little slow because I didn't spend much time in the office. There were a lot of things that needing doing around the house, and so most of the day was spent away from this wonderful playroom. It's nice to be back this morning.

This evening I got word from Joel that the ISP guys had finally gotten around to installing GTK 1.2.8 development includes and libraries so I could get back to building CIA on their machines. Thankfully, it built fine, so now I'm in the process of transferring up a few slides so that I can test the speed and overall functionality. It should be fine, but I have no idea as to the speed issue. My guess is that it's reasonably fast, but nothing to write home about.

I also spent a bit of time today writing out my goals/plans/etc. for CIA and sent them to the rest of the guys. I'm hoping that this starts the process to either get serious about CIA or get out. I don't really care which, but if they wait too long I'll decide for them in what I do professionally.

UPDATE: I did a similar run on sparky and the ISP hosting the CIA website. I used the Dako #6 run with the following options: '-op -om -m' so as not to impact the FFTW time on one machine and not another. What I got was:

Which means that the ISP is running at about 28.2% of sparky. This is nice to know so that I can get a reasonable idea of production runtimes by looking at the runtimes on sparky and then multiplying by 0.282. Not like it's brain surgery, but it's nice to know.