What's it all about, Alfie?

This afternoon I had to deploy an inherited app again, and this time it was a little smoother, but the fact that database changes made in one site aren't also done to another really bit me in the behind.

I had asked one of the DBAs to build this stored proc for me, and he did a great job. What I failed to realize is that he stopped right there. He didn't put it on the London copy of the same database, assuming, I guess, that if I'd wanted that, I'd have asked for it.

Silly me.

Well... it was a little harried and hurried to get things in place, but in the end I put the stored procs where they needed to be. Lesson learned, Bob... make sure to do it yourself.

I've seen this pass around late last week, and finally decided that it wasn't a big issue to get it and see if it's as nice as they say. Certainly, it's more for ObjC developers, and boy do I wish I was one of them, but not today, and maybe not for a while (bad day today). It's nice to see the guys from Panic giving like this.

Anyway, it's here and probably worth getting even if you use it only occasionally.

Today has been a challenge for me. I haven't had a really good day, but I have to admit, in the grand scheme of things, it hasn't been the worst either. It's been a day where I've been fighting my instincts to do "more" - a kind term for stepping on people's toes. More VM problems - I've just had it with VMs for web servers - I've asked for dedicated machines for the web servers (UAT and Production).

I've also run into a most 'unique' situation with chat. Seems we had a custom chat system created - based on IRC, but extended to allow for all the compliance issues that might arise in a trading shop. No surprise there.

No, the surprise is that we have no documentation on the authentication system at all. It "just works", but if it stopped working someday, there's not a scrap of documentation to say what it should be doing as a place to start to see what's going wrong.

This lack of design docs bothered me - I just wanted to be able to code it up so that my web app with alerts could chat them to the users. It's a simple thing, really. But you would not believe the kind of run-around I seemed to get.

Some folks just said "don't use it". Well... it's a little late for that - it's in production, not going away, and I need to chat to something. Saying it's horrible is a judgement call, but I still have to work with it. I just want to know what it's doing!

Several hours have passed and I've just asked over and over to talk to someone at the shop that built this. I just need to know what it's passing around and I can take it from there. It's not magic, it's code. But rather than try to guess looking at the code, I wanted a simple design doc - or even just a protocol explanation.

Hours later and nothing. I'm guessing this won't be over soon. I have no idea why.

It's just very draining. Very. I'm tired and I want to go home.

They found a crashing bug that's been in the code for a while and so issued a fix for that right away - snapshot 47. Nice guys. Thankfully, the crashing bug isn't something I've run into, but it's nice that they are really pushing the envelope of user testing and finding these things.

Today was a day where I was coding all day long. In the middle of the addition to the code so there wasn't a lot of design, and not really near the end where there were a lot of tests. It was a day where I was just heads-down coding.

There were a few tests in the middle - when I got to the point that I was able to send emails from my alert system. But those weren't nearly as interesting as the conditions I could check for were pretty simplistic and I knew there needed to be a lot more complex ways of looking at the data before I was done.

Not a lot to say... just coding.

Well... today I lost about an hour messing around with some code I had refactored to make a little factory of sorts so that it could be shared amongst two classes that had no common class ancestry. It was that or a lot of copy-n-paste, and there was just too much code to do that. So I did the refactoring and then a lot more coding and then hit the tests.

It wasn't the coding, but the testing that gave me fits. Specifically, because I did so many changes, I just did the standard Ant build and then deploy to the Tomcat server. There was a problem in setting this trigger on an H2 in-memory table, and for the life of me, I could not figure out why.

So I dug into the code, and what I found was that I was the victim of incomplete dependency checking by Ant. I should have rebuilt a few more classes that were effected by the refactoring, but Ant wasn't smart enough to do that. I'm not complaining, but when I look at Ant, I really see something made by a group of people that didn't like Makefiles.

I've used Makefiles for years and it's a far better system than Ant - even for Java, and while it doesn't claim to do dependency checking on it's own, there are tons of tools to create those dependency checks that are air-tight in their completeness.

So once I did a clean build, things started working again for me. It's something that I've had long conversations about with friends - the way the Java Lobby seems to be unsatisfied with the existing state of things, and rather than just make a 'makedepends' for Java, they have to XML-ize it, and make a new Java-based, Java-built tool that claims to do these things, when it really only does them slightly worse than the 20-year old tools that everyone else knows.

It's what I dislike most about Java. Not the language, but the people that are it's fanatical following. Spring, Faces, J2EE, the list goes on and on.

Today I needed to get really busy on the next phase of my web server/services - intraday alerts. Basically, we need to have a system whereby it's relatively easy to configure multiple alerts based on the data we collect about the company during the day. The first test is a simple one - if the P/L of any portfolio jumps more than a $1 mil in one tick - either way, then it's something I need to know about because it's likely bad data.

You can think of a lot more business-useful cases where it's not the data that's wrong, as in this first case, but that is going to bring into focus the moving average code I wrote as well as a much more complicated set of rules than simple edge-triggering. So what I wanted to do was to make a system a lot like we had with InfoShop in my last job: basic tools all wired together, but not configured until they are put in place in a configuration file.

I like the Java Properties file format for this, but primarily in it's ability to dovetail nicely with the System Properties and overlay multiple files, one on top of the other. The idea is to have the property files specified in the web.xml file as a list where order matters, and the list is loaded and layered on top of the System Properties so that the complete configuration of any component can be parsed from this map.

Then, I'll have as one named property a list of names. These will then be added to the 'baseProperty' name - something used to keep the Property namespace clean, and these will then be parsed and created by the system. It's a pretty easy way to have a config file do all the building and running you need while not having it be as overly complex as say, Spring.

My initial file looks like:

  #
  # List all the named alerts to create
  #
  FC.Alerts=CrazyJump
 
  #
  # CrazyJump - send email on big jump
  #
  FC.Alert.CrazyJump.ClassName=com.chicagotrading.firecache.trigger.BasicAlert
  FC.Alert.CrazyJump.Portfolio=[*]
  FC.Alert.CrazyJump.AlertExpr=abs((pnlSumNet - lastPnlSumNet)*fxToUSD) > 1000000
  FC.Alert.CrazyJump.MailTo=robert.beaty@chicagotrading.com
  FC.Alert.CrazyJump.Message="Portfolio " + portfolio + " jumped"

And from this, it's pretty easy to see where the name is used, and that there are a few mandatory properties for each named alert - the ClassName for instance. But there can be a lot of optional ones as well - dictated by the class that is functioning as the alert.

This is where the BasicAlert comes into play - we needed to build it with all the capabilities a general alert would need. Like the ability to parse expressions... send emails... create messages. All these things will be needed by the subclasses that will be the far more interesting alerts to write with far more complex business logic.

Still, it'll be relatively easy. We have a simple init() method that will be called, and then an onUpdate() for each updated row to check. It's a pretty simple pattern that's been used a lot in Java, which is why I used it. There's almost no Java coder that won't recognize this when he sees it. Consequently, it'll be easy for just about anyone to pick up and make an alert.

It took me the majority of the day to get this to the point that I think it's ready to use. Lots of set-up for the uses I'm trying to see will need to be covered. Thankfully, I've done something like this a couple of times before and have a really good idea about what needs to be done for the vast majority of the cases, and a few little things that will make even the odd-ball case possible.

I'm tired, and that's the way I like it. Leave it all on the track. Hold nothing back.

It's been a while, but this morning I'm happy to say that the crew working on MacVim have been able to put together a new update of the fantastic editor for the Mac - Snapshot 46. There are quite a few changes including NetBeans support and a new logging/debugging facility so that should we have any issues, it's much easier for the guys to debug the issue.

Fantastic editor. If I only had one, it'd have to be Vim.

Today I was faced with a nasty little problem. I've started to add the annotations to the Google Visualization AnnotatedTimeLine widget and when switching them on and off I ran into a problem with the way I had implemented double-buffering: the unused area where the annotations are in the Flash component has a transparent background, so given that the two widgets have the same size in pixels, the "hidden" graph that doesn't have the annotations active is going to "bleed" through the one that does, with it's transparent background.

Yuck!

How to fix it?

Well, the simplest idea I had was to place a curtain between the two divs in zIndex space. Basically, in the HTML I added another div with nothing on it and set it's zIndex value to midway between the other two: 25.

This wasn't working right, so I changed the CSS on the curtain div to have a background-color of #fff. This looked a lot better, but again, it didn't seem to work all the time.

I pushed and prodded on this... adding a non-breaking space to the div... changing the color to try and see what was happening... then it hit me: Maybe the zIndex was being changed?

I added the one line to reset the zIndex of the curtain to 25 every time I flipped the other two, and Bingo! That was it. It seems that at least in Firefox, the JavaScript/CSS was "moving" the "static" div in zIndex space when I moved the others. Odd, but at least I have it worked out. Sweet.

My web app caches the data it sends to the clients so that if another client asks for the same data while nothing's changed, I send them the same data and it's pretty fast. The problem there is that it's timed data, and that means that all the clients around the world have been seeing the data in Chicago-time. Not really what I want for a good user experience in New York and London.

So I needed to see if there was a way I could bring client localized time to the app without losing all the advantages of the cached datasets. If I passed in the location (region/TimeZone) of the client to the servlet, then I'd have a different cached response for each request in a region. Not ideal. Much better to be able to do it client-side. But that begs the question of how to do this?

One way would be to have the server generate GMT times for everything and then each client would have to deal with adjusting the data to the local time, and I almost went with that, but I decided to stick with the Chicago-based time and then add the GMT Offset in the package from the server.

If the client has the GMT Offset of the server, then it can see if it's GMT Offset is different from the server's, and if so, it can calculate the difference and apply it to all the times in the result set. Nice... for the Chicago users (the vast majority) there would be no change as the changes would be skipped, but for New York and London, I could update the data.

OK, how to do this?

Well... the first part is to generate the GMT Offset in the servlet. Since the offsets in Java and JavaScript have an opposite sign, the right thing to do is to standardize on one and then make the other 'fit' it. I chose to stick with the JavaScript version, so my Java servlet code looks like this:

  public UserInfo() {
    // don't forget to call the super's constructor - no matter what
    super();
    // start by getting the GMT offset in mins for this guy
    GregorianCalendar   now = new GregorianCalendar();
    if (now != null) {
      _gmtOffsetMins = (int)(-1)*(now.get(now.ZONE_OFFSET)
                                  + now.get(now.DST_OFFSET))/60000;
    }
  }

where my UserInfo is passed back to the client AJAX with some information about the validated user. Now it's got the GMT Offset in the same sign as JavaScript expects. Good. Now all I needed to do was convert the data that needed converting.

I created a JavaScript function:

  function convertTimesToLocalTime() {
    // get the time right now for the correct offset
    var now = new Date();
    var localOffset = now.getTimezoneOffset();
    if (localOffset != userInfo.gmtOffsetMins) {
      // get the soze of the table as we're going to need it
      var rowCnt = graphData.getNumberOfRows();
      // now let's run through the first column anc update the times
      for (var r = 0; r < rowCnt; ++r) {
        var obj = graphData.getValue(r, 0);
        if (obj instanceof Date) {
          // get the time from the server into UTC
          var utc = obj.getTime() + userInfo.gmtOffsetMins * 60000;
          // make this UTC time into a local time
          graphData.setValue(r, 0, new Date(utc - localOffset * 60000));
        }
      }
    }
  }

With this, I don't do anything unless I need to, and when I have to, I get the time data in the local timezone - regardless of where that is! I'm really happy with this as it adds a little overhead to the client, but it allows my server to not mess with localizations and keeps the cache intact. Nice.