What's it all about, Alfie?

We have been forced into updating a Data Vendor's API because they never really envisioned that we'd use their desktop product in quite the way that we had - which is to say, built a server around it and made the content accessible to a large number of applications for the traders. So, when they saw what we did, they decided to make that unsupported, and brought out a product to be used in the same manner - for considerably more money. Surprise!

So, we had to upgrade, and in this upgrade, they biggest changes are in the authentication and permissioning of the data. It's understandable, they need to protect the data from unauthorized eyes, etc. The problem I'm running into over the last few days is the way they are having me (and all the other API users - all eight of us) do this authentication. First, everyone has to have a login to the service. Seems that's a good place to start and end. But no. They have an underlying 'ID' - that's guaranteed unique. Seems that's a good place to end then. But no. The minimal authentication is done in their Java client with an ID and the IP address of the machine they logged in on.

Now it seems to me that they already know this, and all they really need is the login. The login leads to the ID, and the login tells them the IP address of the machine they are logged in on. The way they have done it is doubly redundant. That's taking a bit of doing, but it gets even better in the C API.

In the C API - if I'm reading the documentation correctly, you need not only the ID, but four other pieces of information that are known by the service but I have to provide and have no way of knowing programmatically. I have to have the user go to a page on the service's application, read off four multi-digit numbers and then feed them to the C call.

That's an embarrassment. I can't believe they think this is a good scheme. I'm hoping that these four are redundant, or unnecessary, and the docs are simply out of date. But I have a feeling that they aren't and this is just the "way things are" with this vendor. I'm really shocked at this because I had a pretty decent way to have my market data server incorporate this into it's core so that data coming out of the server could be filtered based on the requested user. That would be wonderful as it would put all the permissioning behavior into one place which makes it far easier to control.

We'll see what I get back from one of their "API Experts". I'm really really hoping I'm wrong here.

UPDATE: I got a little help from one of the experts, but for the most part I have to say that getting bad information is worse than getting no information - which was the case today. I asked for the parameters I needed to pass in and got back a list that wasn't right. Once I figured out it wasn't right and what might the right set be, I was well into the afternoon and yet I did get it right. Then I had to realize that their idea of returning a negative was returning an error and not a negative at all. So I had to code for that as well. It's amazing... the people who created this additional functionality didn't even study the old style of doing things and it makes for a very clunky interface to the system. It's working, but as I had expected going in, it's not easy and it's a pain to make it look pretty.

Tomorrow I need to wrap this authentication into a request class and then integrate it into the market data server and finally add caching on the results so that multiple requests in a day don't need to each go to the service to be resolved. Once a day is going to be enough.

A few days ago, a couple of developers stopped by to ask me if I'd seen a problem with the VantagePoint graph initialization when used in applets. Basically, an applet built using the VantagePoint graphs would initialize quickly (sub-second) when only one core was available to IE. Add a second core, and the initialization time stretched out. In fact, adding successive cores stretched out the time so much that at four cores the initialization time was measured in minutes. Clearly, there was a problem. Their initial assessment was that the locking and unlocking of the VantagePoint data tables was the culprit, but I told them that knowing the guys who built it, I doubted that - but I sent them an email just in case.

Today, one of the developers gave me an update and it turned out that it wasn't just IE. Firefox had the same behavior. The other developer had created simply HTML files that would show the problem. So I decided to get that file and see what I could see for myself.

Interesting problem.

It turns out that it's not the locking and unlocking, it's the redrawing that's done after the unlocking that's causing the problem. What was happening was that every additional data point added (parsed from the applet's param tags) was causing a redraw, and Windows was trying to slice things in between the redraws and that was causing the slowdown. Even after I put a lock around the entire data parsing, the delay simply moved to the parsing and setting of the graph's structure.

The solution was to hold-off on all redraws while parsing all the applet's param tags. Then, after it's all set-up and ready to go, unlock the graph and let it redraw the viewport once. This made the four core initialization faster than the old single core because no redraws were being done at all. Since this is a time that you really don't need to look at the intermediate results, it seems like a reasonable compromise.

What amazes me is that Windows isn't smarter about this. When multiple cores are available, Windows should improve performance not have it get worse. This might be as simple as upgrading the video drivers - who knows, but the fact that an applet in a browser is causing this kind of problem for Windows is amazing. Anyway... the code in BKit is fixed and the developers are back at it loading faster than ever.

I learned a new thing - at least it appears to be new, about crontabs on Solaris 8 - maybe later versions as well. It seems that if you have blank lines in a crontab it's going to give you errors. This is news to me for I've had a crontab on a Solaris 8 box for about 4 years that has blank lines in it and it's working fine. But today I had to update my crontab on my main Solaris development box and it was giving me fits about an illegal end of line.

I tried a ton of things, finally I dug into the man pages on the Solaris box and it said right there that blank lines should be avoided. Odd. I looked at another Solaris box and it's crontab has blank lines and it's working fine. But sure enough, take the blank lines out of the crontab and it's working fine.

Odd... but at least I know what to avoid on Solaris.

I know I shouldn't be surprised given the recent history I have had with software Vendors recently, but today was once again an eye-opener. I have known that an upgrade of a certain vendor's API was coming for many months. It took a lot of negotiations, explanations of the usage patterns, etc. to get to the point where things are all signed and executed so that we can actually get the software. Today was the day we could finally start the process of actually getting and using the new API.

It started out with the same problems I've come to expect from this vendor - the software was for Windows, Solaris, and Linux - yet in order to download it, you had to be running a Windows client app. Why? Why not just have a password-enabled web site where you can get the files? It seems that when you're charging what these cats are charging, you can invest the modest sum to get someone on your staff to figure this out. Nope. It was a sign of things to come.

After getting the packages I asked the 'installation specialist' what I needed to know during the install. What questions would be asked, etc. so that I could have all the proper data available to me before the install started so that I wouldn't have to start over or back up during the process. Also, I like to have all this documented so that when I do it on the next box, I'm ready to go and the delays will be minimal.

'No problems, it's self-explanatory' he says.

Wrong. Several times I had to email him to get the specifics of what the question is asking for - things like the version of the API to install? What's the difference in the versions? Why choose one over the other? Things like this are essential in an install app, and there's no documentation - nothing, to help.

I get him on the phone and work my way through the install questions. Then I'm passed off to an 'installation technician' that is going to assign a login and certificate for the installed software. I'm asked if I have my key fob - What key fob?

Clearly, they have forgotten to tell me I needed this, what role it plays in the software, and so on. It's a ridiculous situation. A simple text file would have explained all the steps and why each was necessary. I'm not sure if I'm the first that's asked these questions, but I'm betting not. Why I had to get as many people on the phone and have them explain these things to me I do not know. In the end, I'm not really sure that I have the real explanation, but it appears that the software is working, so I'm willing to take a few things on faith. But how easily this all would have been avoided with a simple installation guide.

Once I got that working, the production box running the old version of the software stopped working! I was not surprised. They had made so many mistakes like this in the past that it wasn't an amazing surprise that by turning on the new version on the development box, they disables the old version on the production box. I had to call back and yell that they had taken my production box offline and they needed to get it back up ASAP. 45 mins later, it was back up, and I was able to tell my users things were back to normal.

The software upgrade itself is a welcome upgrade. There are fewer libraries to link and they are all shared libs as opposed to static libs - which is nice. So it's clear that someone was thinking when they did this, but it's also clear they didn't give a lot of thought to the converting users. There were no code changes - just a recompile and re-link and we're good to go. But what an ordeal. It's amazing what some vendors call support these days.

Making the rounds this morning is an implausible idea that the Twitter downtimes are actually planned to keep people 'wanting more'. While drug dealers have used this in the past, to great success, I don't think the geek-mentality would think this trade-off on the pride of creating a stable, useful system would be worth it.

Take a look at Slashdot... the ultimate geek hang-out. If they went down, it's a scream-fest and they loose credibility - not gain it. Twitter is a communication channel, no one thinks the phone company is good for killing service. Nope. This is something they are trying to prevent, and simply can't. I'm not sure why, but it's got to be that there are things going on behind the scenes that the casual user doesn't see.

The excuse for the latest outage is a database outage due to a routine update going bad. Now this I know about... if you're updating a 'production' database during working hours you need to make sure that you have a back-up, or a fail-over, or something so that if, on the one-in-a-million chance that something goes wrong you aren't out of business. They messed up on their disaster/recovery plan. Maybe it's not big enough to have one of them, but I think it's time they think about it.

In the end, it's nice, and people will use it, but they'd use it more if it were more reliable - not less.

I read this article this morning and from the sounds of it, Intel is getting closer to getting it's quad-core into an operating mode where it'll be able to be dropped-into the MacBook Pro frame and just run. While the report estimates that the shipping date would be sometime during the summer, it's unlikely we'll see anything from Apple until the fall, even if the chip ships from Intel on time. Apple has it's hands full with the iPhone and iPhone Store on iTunes.

However, it's very exciting to see that they will be producing something this year that should make it to the MacBook Pros by early next year at the latest. That will be an exciting thing to see.

Well, I have to say that this Kindle is awesome. It makes carrying books a dream. I got a sub-$50 4GB SD memory card and now it can hold more than 3,200 books - based on their estimates. That's a ton of books. The battery life is great - I haven't had to recharge it for days after turning off the wireless access. The page flipping is not a big deal to me - a simple flash, and it's the new page. The print is crisp and legible - I can read it as easily as a paperback. It really almost makes it fun to read. Not that reading wasn't fun, but this just makes it exciting.

It's a wonderful little gadget. I'm very glad I got it. I know there are downsides, there always are. But if you believe that Amazon is going to be around for the long haul to support this, then it's a great way to get, carry, and read books. Highly recommended.

Acorn 1.2 is out, and it fixes the one bug that I had found: multiple magic wand selections to create the exact selection you wanted. In the previous version, for each additional selection you did, the image seemed to move down a bit. The result was that you really couldn't do the multiple selection properly. I had to resort to other techniques, which worked just fine. In the latest release, I'm glad to say that that bug - along with a ton of others, and quite a few new features has been added.

I once read that Acorn is a programmer's editor, and to that I agree. I'm not a graphical artist, but I know what I want for images, and Acorn makes it exceptionally easy to get what I need in and out. Wonderful app.

Given the state of web services and protocols these days, it's really pretty amazing that Twitter is as unreliable as it is. Before I even started using it, I had read many reports both praising the idea and cursing the implementation for it's unreliability. New things can start out on shaky ground, but as time goes on, they should stabilize out and get more reliable. But I don't see that happening with Twitter. I'm using Twitterrific and while I like most parts of the app, and understand they are working on an iPhone app for release in a few months, I have to say that I don't for a second think it's Twitterrific's fault.

So why is Twitter so unreliable? Can it be the load? Possible, but that's something that can be scaled up, and given all the attention it's received, I'm guessing that they aren't hurting for money to buy additional servers. So I don't think it's a matter of hardware.

Could it be the idea behind it? Doubtful, IM clients handle much larger packets of information and they aren't this bad. Also, given that it's not a 'push' technology, it's hard to understand the possible issues. Maybe it's the SMS or IM integration issues... don't know.

I've heard that it's all about being first. I disagree. That's a big factor, but it's not the end-game. Look at VisiCalc. Can't? Exactly my point. They created the spreadsheet and Microsoft beat them so badly that now most folks won't even know their name. Twitter? Twitter who? That's what might happen if they don't get the reliability up. People will like the idea and the barrier to entry is nothing for a Google, Yahoo, or Microsoft.

Ideas are great, but eventually you have to deliver. Twitter needs to get busy and deliver on the reliability.

When I got home today my Kindle was waiting for me. I'm currently in the middle of Game of Thrones by George R. R. Martin - about 120 pages in, so I didn't think I'd be using the Kindle until I finished off this 800+ page book. Amazon, and luck, proved me wrong.

What I think one of the nicest parts of the Kindle is that I can get recent releases at paperback prices. I'm not one to buy a hardback book for myself. I just don't. I wait until they come out in paperback and then get them. That means I'm about a year and a half behind 'current'. For about the price of a paperback, I can get new releases on the Kindle. That's really nice.

So I open up my Kindle and find out that it's already been set up for me. That's probably nothing for Amazon to do, but it's an amazing feeling to have it there, out of the box, and ready for me. Very cool. I read the introduction on the Kindle - on the Kindle, and then decided to check out a few books I've wanted to have 'on hand' to read. So I go to the Kindle store.

Very nice experience. Clean, decent. All I could have expected in a wireless delivery system. Very nice.

So I look up Clive Cussler and get his latest two that are out in hardback. It's next to nothing to get them, and $20 later, I have them. It's simple, fast, and enjoyable. I can see why people are saying this is the 'iPod of books'. It's not a perfect device, but it's darn close, and they have a very nice online store.

I then go to look up Goerge R. R. Martin and find out that I can get two eBooks - the one I'm reading and the next in the three-part series for the cost of the second one. I get them both. I then use the Table of Contents on the Game of Thrones to get to where I am in the 'dead tree' version. My Kindle is ready to use.

Amazing experience. Really. About the only downside I've seen is the carrying case. I'm not sure that I'm going to use it - I don't use one for my iPod Touch and just take care of it, but it would be nice to be able to 'lock' the Kindle so that accidental presses didn't mess with things. But maybe they can come out with that in a software update.

In any case, I can see that my reading will be changed forever. I'll pick up a 4GB SD card this weekend and plop it in the Kindle and be set for life. Really. I can't imagine ending up with more than 4GB of books on this thing, but that's a staggering thought. I only wish there were a way to get my physical books on the Kindle without re-buying them. That's the one beauty of the iTunes system and music. I can convert them by ripping. Can't do that easily with books. Shucks.