What's it all about, Alfie?

Things are really heating up between some groups in Google and some groups in Apple. From what I've read, it seems like it's more of an Executive War, as opposed to a real "can't stand to look at you" war in the trenches. But hey, that's what makes the game of Executive Watching so darn much fun.

Well, this morning I was reading Slashdot and saw that Google announced that they would be working with Adobe on a new plugin API for browsers to support Flash, and including Flash in Chrome. This not only covers the initial install, but also any subsequent updates. This is a major slap in the face to Apple and it's push for HTML5 as the technology moving forward.

In a sense, I'm really happy that Google is doing this. I use Google's Visualization widgets daily at The Shop, and a better Flash in Chrome is going to do nothing but help me. That's fantastic. But in a larger sense, I think the better solution is to see the components I use ported to HTML5 Canvas or SVG graphics like some of the Visualization widgets.

My ultimate goal is to have a visualization platform that does what I need without crashing, or bringing the machine to it's knees. CUrrently, I have to do a lot of little "tweaks" to get a reasonably stable platform, and even that's a bit dodgey at times. Anything that Google does to make this better is a good move in my book.

That being said, if I saw that someone came out with an HTML5 Canvas-based graphing package like the Google Visualizations, I'd immediately check it out. If it had a better system of support and less problems in the deployment (I can't deploy on Firefox, and IE only with the Google Chrome Frame plugin), then I'd switch over and never look back.

I don't think Google is doing the right thing in the long-term, and that's what I've always thought they would do. Engineers giving into silly Executive battles is just silly. It's beneath them. Beneath us.

Stop it. Convert the AnnotatedTimeLine to HTML5 Canvas, and then stop messing with Flash. That's a winning strategy.

For the past several years, Sun's Solaris for SPARC (at least) was "free" - you had to pay for the media and shipping. In that time, I bought Solaris 7, 8, and 9 for my SPARCstation 20. It was a fantastic program as it made sure that people like me would still be running, using, and liking Solaris, long after the linux wagon came to town. Had Sun charged me full price for Solaris, I'd have been running linux on that same box all that time. It was a good move on Sun's part to have this program in place.

Then Oracle bought Sun... and every good thing must come to an end.

This morning I was reading Slashdot and saw that Oracle is suspending the free licenses of Solaris. That's sad.

The funny thing is - I doubt it'll generate much revenue for the company. But it's going to cost them a lot of potential adopters in the kids that get old machines to play with and now won't learn Solaris and thus keep Oracle out of the long-term business. It's just not smart.

But then again, I've never thought that "smart" was a good adjective for Oracle.

This morning I noticed that OsiriX 3.7.1 was out with an impressive list of fixes and features. It's not like I'm a radiologist, but I do like being able to see the slides of the family. It's just kind kooky.

This morning I got a tweet saying that CoRD 0.5.4 was out, and as long as I might need access to a Windows box at work, this is the best way I've seen to make it usable on the Mac.

This morning I saw that Miro 3.0 has been released. It's the open source video access tool that can download content from a ton of places and update subscriptions, etc. similar to the iTunes Store for Open Source content. I first started looking at Miro when I wanted to be able to download episodes of House, M.D., but it turned out these were only available on Hulu.

Anyway, with this update, there are lots of interesting things but the question still remains: is it a better way to watch House? Don't know that's the case.

A few days ago I ran into a serious problem with my usage of the H2 database as an in-memory database for a ton of data. Like we're talking nearly 70 million rows in a day. It's a lot. The problem is that I need to be able to INSERT this same data into (optionally) two databases. This makes is most logical to build up a (large) SQL string and then send it to one, or more, databases through JDBC. Seems pretty simple.

Until it gets too big.

My errors were because I was adding a lot of data to this table. So much that it was blowing out the JDBC Connection. So I put in "breaks" into the SQL string and then chopped up the transactions by those breaks. What I didn't think about then, and did think about this morning was: What if I broke up the statements?

So I tried that.

  // break up the SQL into it's component statements
  String[]  parts = null;
  if (!error) {
    parts = sql.split("\n;");
    if ((parts == null) || (parts.length == 0)) {
      error = true;
      log.warn("Unable to break up the SQL into statements!");
    }
  }
 
  /**
   * Now let's make the JDBC Connection and Statement and then
   * execute each of the statements against it.
   */
  try {
    ...
    for (int i = 0; i < parts.length; ++i) {
      stmt.executeUpdate(parts[i].trim());
    }
    ...
  } catch (SQLException se) {
    ...
  }

I broke up the large SQL by the statement terminations, and then ran through all the statements one after another, until the complete package was done. The effect was dramatic: It worked!

Not just that, it was far better because now we were able to once again maintain the transactional integrity of the INSERTs so that I could use these tables with one another without having to worry about when certain rows were inserted into certain tables. Much better.

So it may still be a bug in H2, but it's easy enough to work around.

This morning I noticed that Perian 1.2.1 was out with several bug fixes and supporting a new encoding. It's a great add-on for Quicktime, and it makes it a lot easier to look at unusual content in Safari. Nice to see the addition.

This morning I noticed that Firefox 3.6.2 is out and contains the requisite number of security fixes and performance enhancements. It's just another "point release" for the Firefox team, but good that they are making progress. I'm very interested in the new JavaScript engine they are planning, as I'd like to see Firefox overtake Google Chrome in that area. But until that day, I'll deal with these little improvements.

Today I've been fighting a bug in the H2 database that has been quite a puzzler. I'm not being overly critical of H2 as I think it's a great open source project, but when you see how non-obvious this bug is, I think you'll agree that there's a lot that still needs to be done with H2 in this regard.

To start, I am using H2 as an in-memory database for a web app that is collecting a lot of data - working on 9 million rows in memory. Not small. But the problem isn't about it's size, it's about a certain series of database inserts that cause, of all things, a Java StackOverflowException. Let's start with the table.

It's one of many tables in my in-memory schema, and it's pretty darn simple:

  CREATE TABLE volatility (
    acquired        datetime    NOT NULL,
    product         VARCHAR(64) NOT NULL,
    expiration      DATE        NOT NULL,
    strike          FLOAT       NOT NULL,
    volatility      DOUBLE,
    generationTime  datetime NOT NULL,
    CONSTRAINT volatility_PK PRIMARY KEY (acquired, product, expiration, strike)
  );

This guy is getting hit with all the volatility data we have for a given portfolio - inserting hundreds of rows every few seconds. The primary key of acquired, product (or underlying), expiration, and strike make it very easy to look up this volatility data for just about any where clause I might write.

When I started putting together the SQL string for inserting the data, I started running into problems. Specifically, I don't use PreparedStatements because I can't be sure how many inserts I'll be doing and there's also the problem of being able to optionally do the same updates to the back-end database if the configuration is set-up that way.

I have my UAT box writing it's data to the back-end database, and all the others just read that when restarting. This flexibility requires that I have something that travels as an object, and can be used to update one or two databases without having to re-create anything. This leaves out the PreparedStatements as they are built off the connection, and that's database-specific. So I had to stick with straight SQL.

In general, that's not too bad, and I haven't had any problems with this scheme until today, but today I think I met my match.

What I found was that when I started feeding the system the data for the volatilities, I could have it all run smoothly if I limited the number of strikes to five or less. But if I limited it to 50 or less, or no limit at all, I'd get a SQLException with the message being that there was a 'general' exception: StackOverflowException. I had no idea why I had created something with excessive recursion, so I went through the code several times and couldn't find anything.

I saw that H2 had been updated to 1.2.132 (from 1.2.127) and thought that maybe it might be a bug in H2 that had been fixed. Don't get me wrong, this is a bug in H2, but pin-pointing it to some specific code snippet is not going to be trivial. If I have the time in the future, I might do it, but it'd be a long-shot and take the better part of a day to get the test data set up into a simple test app to send to the H2 developers.

Anyway, I got H2 1.2.132 and tried it - no difference. Crud.

So then I tried setting the auto-commit on the JDBC Connection objects to "ON". Initially, my code had it set to "OFF" with an explicit commit() called on the connection once the SQL statement is executed. But even with the auto-commit "ON", we had the same problems.

Next I tried breaking up the SQL itself into smaller blocks based on the number of volatility points per expiration. If there are more than five volatility points in an expiration, then I break the SQL into separate statements in the execution part of the code. Since these are somewhat stand-alone, it shouldn't cause any problems. But I had my doubts.

I was really surprised to see that this break-up based on the number of strikes for an expiration solved the problem completely. What a break!

What I don't understand is why.

I can see that 4 of the 6 columns are in the primary key - but that in and of itself shouldn't be the issue. Not for a recursion problem. I can see that I'm inserting several hundred of these at a time, but the problem appears somewhere below 50, so that doesn't make a lot of sense.

Nope, this is a real puzzle to me. If I get the time, I'll try to write a simple test app and ship it to the H2 developers to see if they have any idea as to why it crashed. Very odd indeed.

This morning Google Chrome updates went out on all release branches, and for me that means I was updated to Chrome 5.0.356.0. I'm still hoping that they soon pass up Firefox as my secondary browser, but they haven't been able to do that - yet. I'll try to give this new release a go and see if it's gotten good enough.

Eternally optimistic... that's me.