What's it all about, Alfie?

Today has been spent primarily on getting the PRIVMSG capability working on this custom-developed chat system. Because of the nature of the system, that is not allowing private chats, it was a lot more difficult than it needed to be. While I might have done things differently in the implementation, I wasn't involved and this system is in place and there's very little interest in really looking at this critically. It's "done", it "works", let it go.

I had to work with what I had.

That included the sum total of the code. It was a combination of C#, Python and assorted scripts and was virtually undocumented. It was a mess. But as far as messes go, it was something that I could dig into and get a few hints.

My primary debugging tool was the log of the socket data coming from the IRC server. It was literally invaluable. In the end, this is the interchange I discovered I needed to use in order to get person-to-person chatting, and therefore code-to-person chatting, working.

First, my code had to issue a special command to a particular "Overlord" bot that controlled the users on the IRC server. This in and of itself was interesting. They chose not to implement the rules in the server, but rather in a Bot that existed alongside the server.

When I send the appropriate command to this bot, I need to include the nick of the user I want to talk to. The Overlord then creates a channel just for the two of us with a special prefix, and the two user's names separated by a special character. Both users then get INVITEs to this new channel.

I have to wait for the INVITE, parse the channel invitation into the two user names, see which one is me, and which one is the "other guy", and then cache this data so that when I want to chat to the particular user, I can look up the special channel for this guy, and chat there, instead.

There were a ton of wrinkles with this scheme. First, because the process is asynchronous, I have to buffer my chats to this person if I need to create this channel. That is a pain, but doable with a little thought. Once I get the INVITE, I simply see if I have any buffered messages for this guy, and if I do, then I send them in order to the (newly JOINed) channel and we're back up to date.

Another one was the reconnection scheme - I simply took the point of view that on reconnection we'll do the minimum and each operation will ensure that everything is set up for that operation. It works, but it might be doing a little more "lazy" set-up than a different approach that would cache the channels, etc.

I have to say that everyone I've talked to in this place about these technical details is amazed that it has been done this way. The simplest way would have been to log everything in the server. Period. Database or flat files, or MySQL (combination of the two) would have worked. Then add in a simple authentication method on the server and you're done. Leave private chats as-is, but log them. Done.

It wasn't done this way, and I'm not sure what's going to happen in the long run. I've got this working, and that's the most important thing.

It's pretty big news today - Amazon received notice from the rights-holders of Orwell's 1984 and Animal Farm that forced them to remove the un-licensed copies from user's Kindle readers. I can certainly understand the folks saying that this is a serious blow to e-books: that they aren't the same as books... you can't sell them after you've read them, you can't donate them to a library, there are a lot of things you can't do. But they say now that you can't even really be sure that they ever really belong to you.

It's a fair critique. But Amazon's business is really the publishers and the customers. They are a middle-man. No way they could exist without both groups. Take one away and the other will leave. There's nothing they can do about it.

So they have to strike a compromise here. That compromise, it seems, is still developing, but it appears that the new compromise is going to be "if you have it, it's yours" - like the Apple AppStore. It's fair, but this also means you can't transfer it to another device - which can be a pain to many as devices fail and need to be replaced.

It's a tricky line to walk, and while I'm not interested in losing my books, I have to understand that when I buy them in this electronic state, where the laws have clearly not caught up to us, that I'm accepting a lot of the risk here. In 50 years the laws will have caught up and this will all be settled - one way or the other. Until then, it's a little bit of the buyer beware.

Sad, but true.

I've been working within the limitations of the AnnotatedTimeLine graph from the Google Visualization API set and have just been telling my users that the unfortunate "flash" to white of the graph while it's updating data is an unfortunate reality of the AnnotatedTimeLine, and that Google knows this is in issue and they are working on it.

Then I read an interesting post to the mailing list about how Google Finance didn't have this issue, and it got me thinking... How could they get around the flash? And in the show this morning, it hit me: Double-Buffering.

Simply have two graphs - one aligned in the z-axis behind the other. In fact, make them have the same coordinates, but just vary the zIndex value to "flip" them one in front of the other. Then, always draw to the rear (invisible) graph and when it's done, flip it to the front by setting sIndex to 49 on it, and setting zIndex to 0 on the one that was in front.

A simple use of an array of two graphs accomplishes this quite easily. I add in two variables - the index values for "foreground" and "background", and then you always draw to chart[bg] and so on. It's very clean and slick. The change of the z-axis values is done in the blink of an eye, it's very hard to see. Compared to the 3 to 6 seconds that the redraw used to take, it's nothing short of amazing.

In a very simplistic form:

  var chart = [];
  var fg = 0;
  var bg = 1;

and then in my initialization code, called from the Goolge setOnLoadCallback() method, I have:

  chart[0] = new google.visualization.AnnotatedTimeLine(chartDiv[0]);
  chart[1] = new google.visualization.AnnotatedTimeLine(chartDiv[1]);
  // tie us into the events for these guys
  google.visualization.events.addListener(chart[0], 'ready', graphReady);
  google.visualization.events.addListener(chart[1], 'ready', graphReady);

and then when I need to draw to the correct graph, I simply call:

  chart[bg].draw(graphData, chartParams);

When the graph is done drawing, I'll get a call to:

  function graphReady() {
    //swap the graphs quickly
    chartDiv[fg].style.zIndex = '0';
    chartDiv[bg].style.zIndex = '49';
    // now swap what we mean by back and front
    if (fg == 0) {
      fg = 1;
      bg = 0;
    } else {
      fg = 0;
      bg = 1;
    }
  }

There's a lot of the details I've left out because they don't really matter. The point is that you have to place them on top of each other, and then do this double-buffering. It's pretty nice.

The folks on the Google Chrome team have a nice download site for the shapshot builds of Chrome for Mac OS X and linux. I can imagine that the linux port will be a tough GUI as the platform isn't really about a consistent user interface, but the Mac OS X port is coming along nicely.

I'm sure it's not even at the alpha stage in the group's mind, but it's up and serving pages. Sure, that's about 10 mins. with Xcode, but the fact is they have the similar GUI to the windows version and it's working. They are making progress and that's nice to see.

In the end, I'm not sure whether Chrome or Safari is going to be the best browser on the Mac. It's nice to have a choice like that, however.

UPDATE: Ah... it's Chromium that has the builds - not Google Chrome. The former is the basis of the latter, but the two are different. Still... you need to have the one to have the other, so progress is still a good thing.

[6/5] UPDATE: Well, that didn't take long. It seems that there is a Google Chrome for Mac OS X out now. Very early release, but it's there and they are working on it. So they only lagged the Chromium release by a few days. Nice to see they are working on it.

Well... they have helped me once again, those Googlers. They have released as stable Chrome 2 (actually 2.0.172.28), and with significant changes in the V8 JavaScript engine they are reporting a 30% increase in speed in JavaScript-heavy pages. Also, with the latest WebKit, page rendering is even faster.

I have to say this comes at a great time. I'm struggling with the size and memory footprint of my web app at work, and I have high hopes that this version of Chrome is going to be more stable, faster, and more memory efficient. Given that the problems are all in Google's hands (Chrome, Google Visualization API), I hope they have made real progress.

I've been working with 29West over the last few days and while I can see it's value, it's a little different than server-based messaging systems, and I can see why it's got advantages, and disadvantages. No need to critique it here... it's just what I have to use. But there's a consequence of using 29West's Java API on Tomcat and that's the fact that 29West's Java API uses JNI to get to the real C libraries under the covers.

With JNI, the shared libraries are loaded once in the Tomcat server, but if you want to change the code and remove and install the app again, you're in trouble because the shared library is not unloaded when the class loader is dropped. There's a lot of unhappy people about it, but in the end, there's nothing you can do. You have to remove the web app, shut down the Tomcat instance to drop the shared library, then start up Tomcat again, and then install the web app again.

I wanted it to be easier.

I got an interesting set of targets to do that. First, I need to have the start and stop targets, and they are simply exec targets to the locations of the startup and shutdown scripts:

  <target name="start" description="Start Tomcat application">
    <exec executable="${catalina.home}/bin/startup.sh"/>
  </target>

and:

  <target name="stop" description="Stop Tomcat application">
    <exec executable="${catalina.home}/bin/shutdown.sh">
      <arg value="-force"/>
    </exec>
  </target>

The value of the argument to shutdown.sh is that if I define:

  CATALINA_PID="/usr/local/tomcat/bin/.catalina.pid"

then catalina.sh will save the pid in the file and then on shutdown.sh it'll do a nice kill -9 on that pid and make sure it dies. This is really important because I need to kill the Tomcat instance and I need it to die right now.

Given that we have the standard remove and install targets from the default Tomcat Ant build.xml file, then all I need to do is glue these together:

  <target name="update" description="Update the application and bounce the server">
    <antcall target="remove"/>
    <antcall target="stop"/>
    <antcall target="start"/>
    <waitfor maxwait="3" maxwaitunit="minute" checkevery="500">
      <http url="http://localhost:8080/index.html"/>
    </waitfor>
    <antcall target="install"/>
  </target>

What's happening here is that we're removing the web app from the Tomcat instance, and then shutting him down forcefully. Then we're starting him back up, but since the startup.sh is asynchronous I need to wait until I can get a page back. When I can, then I'll install the web app again.

All in all, it's pretty sweet. It's not as nice a knowing that it's smart enough to unload the shared library, but there's nothing I can do about it. Actually... now that I think about it, I think it'd be better if the loader was smart enough to see that it's the same bloody file and link into it without an issue. But I'm clearly not as clever as these guys.

What I've got is workable, and given the limitations I have (29West and Tomcat), it's as good as I can expect to do for now.

Today I've spent a lot of time adding a new way of getting data into my web app. The Shop has standardized on 29West, and I've seen it in a lot of the smaller shops, but there's got to be something more to it... cost... performance... something because Tibco is the 800lb Gorilla, but not used in a lot of the shops around town. I'm guessing it's cost. 29West may be a lot more reasonable. In any case, it's different from the majority of messaging systems I've used in the past in that there's no central messaging server. You don't connect to a server and then get a topic. Instead, you simply connect into the system and then find your topic.

It's interesting.

It's using some multicast/broadcast system to talk to the other 29West clients on the network. You connect in a machine-to-machine scheme - publisher to subscriber. It's going to be more efficient with network switches as there's no single point that all messages have to go through. Makes sense. Looks a little like magic to simply "find" the publisher, but that's good coding for you.

If there's a downside, it's in the way you have to set up the communications. It's pretty nasty. You have to create a context attributes and then populate that with configuration values, and then from there you make a context, and then a context thread to handle the socket maintenance so you don't have to do it yourself, and then you find your topic for this context, and then either a source (publisher) or receiver (subscriber) for this topic and context.

It's all needed for maximum flexibility, but really... if you're going to make it look like magic, then why not just do the context, have it take a simple map of attributes - or nothing if the defaults will do, and have it fire up a thread. Then it's find the topic and you're off to the races.

You shouldn't need to make a source or receiver... those should be done in the context, or topic. Send and receive... can't get a lot simpler than that.

I was talking to one of the 29West guys here in the Shop, and it became much clearer to me as to why it was this way: 29West is C libraries and every one of their APIs are just shells onto these C libraries. In that case, it makes sense to have a not-very-object-oriented view of things. Reasonable, and you only need to code it up one.

I haven't had a lot of experience with the performance of it, but I'm guessing I will in the coming months. For now, I'm glad I have test sources and test receivers to make sure I can move messages around - box to box, window to linux, all that. Tomorrow I'm going to have to deal with a few nagging things about the impact on Tomcat, and see if I can't get data from the guy sending it to me to me.

The story seems to be that there have been two recent updates (as in this week) to Google's Chrome browser - one for a security fix, and another for a problem with the 2D image handling. At the same time, I got a call yesterday from some network security folks asking if there was a way to run my web app in the Chrome 'environment' without allowing the users to run the browser itself.

I understand their position - network security is such that an unknown must be considered dangerous, and certifying something is going to take a lot of time and effort. Unfortunately, the JavaScript in the Google Visualization API needs something like Chrome to work well at the amount of data that I'm working with.

So there have been a few updates this week. Nothing major for me, but certainly something to keep an eye on from the network security folks.

My favorite Mac OS X editor - BBEdit, was updated today to 9.2. There are several things I asked for in this release:

• ability to make ctags files from the bbedit executable
• added better function searching in navigation bar

to be fair, there are a ton of changes, and unfortunately, not all of them I'm really happy about. The biggie I noticed right off was the fact that the status bar is now below the horizontal scroll bar, and there seems to be no way to hide the horizontal scroll bar. I need to scroll horizontally, but not all that much. The old scheme was perfect, in my mind. Sad to see it's gone.

After looking at it for a little bit I decided that since I could not remove the horizontal scroll bar, I dropped the status bar. Not what I wanted, but it seems unavoidable.

UPDATE: Interestingly, the file position "appears" to the left of the horizontal scroll bar if you move the cursor. When you switch to another window, it disappears again. Interesting... this is the single most important part of the status bar, and since it's available as I'm moving in the file, I guess it's not really "missing". Great. Kind of wild, but neat.

Well, I heard it was coming, and it's certainly a nice looking device. The Kindle DX is the large-format version of the new Kindle - measuring a full 9.7 inches diagonally. It includes a native PDF reader, and the ability (like the iPod/iPhone) to sense the rotation of the device and automatically go into landscape mode. Other than that, it's the second-generation Kindle.

I'm happy that my Kindle is working so wonderfully, but in a way, that's bad news for me. For as long as it's working OK, I have no reason to get a new one. I think the new version of mine is what I'd get, but it's nice to think about the bigger version and reading books. That would have been incredible in school.

Still... I'm waiting to see if Apple brings out an iPod Touch that's in the same size screen, but color, and smaller overall, and then get that and retire my Kindle. That would be a nice upgrade.