What's it all about, Alfie?

I'm not a big widget fan, but this guy is really rather impressive. Yeah, it's as much that it's PHP, and I'm a big PHP fan, but it's a slick interface to the problem of a language that seriously needs a manual like this. One of the criticism of PHP is that there are so many functions that are database-specific (for instance). This makes one long for a more general function set.

But until then, this is a great little tool to look up the functions in a Dashboard widget.

Today has been a lot of messing around. I'm in the middle of adding a few new features to the app I'm inheriting from another developer and it's been a bit of a challenge. Over the last two days I've been adding the code and the unit tests to the codebase, and today it was time to get it all loaded up into the test server(s) and see how it runs. Things took a little longer than I'd have hoped, but all in all, it's understandable given that I haven't used these servers before today.

A few bumps in the road, but at least they had my favorite cereal in the kitchen this morning. It's amazing how something as simple as what you have for breakfast can really change your mood. There's got to be studies based on developers/creative types where they look at the effect of a new pad of paper, or a clean set of editor colors, or a new font - or cereal, has on your mood and how, in turn, that effects your productivity. In any case, it was a bit of a rough start, made smoother by those wonderful squares of goodness.

So I finally got things running and it came time time to compare the numbers between the code we added, and the application where we're getting the raw numbers. This application also does the aggregation of the raw numbers and that's what we're comparing. Did they match? Some. Never a good result.

I worked the rest of the day trying to get 29West working between a server of data and my app... didn't have a lot of luck, but did find out that speed/duplex problems are rampant even here, and my linux box was plugged into a switch that was improperly configured for auto/auto operation. That was fixed and helped a lot of things on my box, but not the 29West issues.

The data matching tests revealed that the formula I was using for the normalization of the values was wrong, and the right formula was a single factor different - not too bad, really. I fixed that up in the code and fixed the jUnit tests and things started to look better. But still not perfect.

Then the 29West guys noticed that it wasn't my box, but the sender's box that was having problems. So hopefully they'll get on that box and fix things up so he can send around the network. We'll have to see tomorrow.

Finally, my co-worker did the exhaustive testing of taking the raw data and using Excel manually aggregate it and compare it to the data in the app. There's only one problem with one instrument. It was just sprinkled throughout the portfolios so as to make it appear that there was more of a problem than there was. This was good news, but the day was over.

I'm sure glad I had those Mini Wheats.

I can really appreciate the case for unit tests. I have built them in many forms - 'test apps' and simple running tests that exercise the components, all the way up to jUnit tests for Java. There is a place for each, and I can see their value. But anything taken to an extreme can be bad. Very bad.

Take the case of unit tests I ran into today working with the original developer of an app adding a new feature to said app. It wasn't that hard... OK, it was harder than it should be because no one wanted to take responsibility for the data in the system.

OK... here's the problem.

Options expire. Futures expire. Options on Futures expire. But when an option on a future expires on the same date as the future, then the time it expires can be shifted. This is most easily solved by having the expiration date/time a table that understands this. But as with the Y2K issue, shortcuts are often taken in the initial stages of a project, and these things are thought to be "insignificant". Then, several years later (just like Y2K) the fixing of the problem is far far bigger than it would have been initially.

The system we were looking at had the future expiration date/time and the option expiration date/time. I said that the easiest way to know what to do is to look at the dates of these two expirations, if they are the same, then expire the option on the earliest of the future or option expiration. That way, if the future expired first, then the option is dead and has to expire at the same time. Simple.

If the future expiration date/time data were maintained properly in the system. Alas, it is not. Only the date component is maintained. So we had to put in a hack. I hate hacks. This was a double hack in my book because the author felt the best solution was to add a new constructor argument with a map of expiration times for 'double expirations' based on the future. This was then placed in a Spring XML file and then used in the code.

In truth, the change was only about 20 mins to do. We talked about alternatives for much longer than that (Given how I hate hacks). So the time required wasn't too bad.

But the jUint tests... oh... the tests.

We spent the next several hours updating jUnit tests to work with the new functionality. While I'm all for the testing ideas, when the testing code is bigger than the code under test, there's a hint you're doing something wrong. When you find a bug in the testing code, you know you're in trouble.

Testing code is meant to do Unit Testing. Not massive Q/A tests. Those are almost impossible to simulate and there's a reason that they exist. If you're spending two hours fixing up tests with 'fake' numbers, then chances are, you're making a mistake.

I still did it. I believe that consistency is very important - even if it's something I'd fight to the end of time to redo. If it's there, and if you're going to keep it, then by golly... make the new code work like the old.

But when unit tests go wrong, there's almost nothing worse. Almost.

I've been working all day on a single feature for this application at the Shop. It's got potential, but the way in which it's put together just screams 'Unnecessary Complexity', and I, for one, want to put an end to this kind of thing once and for all.

What is it about Java developers that makes them design such systems? What kind? you ask, I'll be glad to explain.

Java, as a langauge, is not evil. It's a tool. No more, no less. It's got a lot of nice features, and just as many limitations. It's not slow, per se, it's just not fast at everything. It's capable, descriptive, and works just fine - as long as you don't ask it to do something it's not meant to do. But that's not the real problem.

The problem, I think, is the way in which Java is evangelized by it's proponents.

Substituting design and planning for scores of small interfaces and "wiring it together" with something like Spring or even the equivalent of the Swing GUI tools, is not the right thing to do. IDEs like Eclipse and the rest allow these developers to put together projects that even they don't understand. This is a real problem.

Case in point. I was working with the author of this package today and asked him where a class was located. I'm a Vim/makefile guy for production software. It's universally available, easily transported, easily used on low-speed lines, and for all these reasons and many more, this is the most efficient toolset I've seen for the complete project lifecycle. So I asked him where the class was so I could load it up in Vim.

He didn't know.

He couldn't remember.

He had to have his Eclipse workarea opened up in order to find this class - that he was clearly very familiar with. After all, he mentioned it to me, and I just asked where it could be found.

This is but one danger - Package Explosion. Dozens of packages that have no reason to exist. He would be far far better served by simply thinking about the project and then laying out a few, well thought out packages and placing his code in these. Having this overly complex package layout is unnecessary, and while it's easy to use in Eclipse, it's a pain even to the developer that created it.

If I could pass these ideas on to this developer, I would. Sadly, he's likely too old to change his ways as he's a strong proponent of this type of development. However, in the hopes that someone someday might read this, here are a few pointers for coding that I've found exceptionally helpful over the years.

• If you can't remember where you put it, the structure is too complicated.
• If you require an IDE, then the project is poorly laid out and too complicated.
• Simplify, simplify... the best designs are the simplest.

I don't think this is exhaustive, but if you can stick to these few rules, you'll be far better off than what I've been dealing with all day. Holy Cow.

Well... they have helped me once again, those Googlers. They have released as stable Chrome 2 (actually 2.0.172.28), and with significant changes in the V8 JavaScript engine they are reporting a 30% increase in speed in JavaScript-heavy pages. Also, with the latest WebKit, page rendering is even faster.

I have to say this comes at a great time. I'm struggling with the size and memory footprint of my web app at work, and I have high hopes that this version of Chrome is going to be more stable, faster, and more memory efficient. Given that the problems are all in Google's hands (Chrome, Google Visualization API), I hope they have made real progress.

I've been working with 29West over the last few days and while I can see it's value, it's a little different than server-based messaging systems, and I can see why it's got advantages, and disadvantages. No need to critique it here... it's just what I have to use. But there's a consequence of using 29West's Java API on Tomcat and that's the fact that 29West's Java API uses JNI to get to the real C libraries under the covers.

With JNI, the shared libraries are loaded once in the Tomcat server, but if you want to change the code and remove and install the app again, you're in trouble because the shared library is not unloaded when the class loader is dropped. There's a lot of unhappy people about it, but in the end, there's nothing you can do. You have to remove the web app, shut down the Tomcat instance to drop the shared library, then start up Tomcat again, and then install the web app again.

I wanted it to be easier.

I got an interesting set of targets to do that. First, I need to have the start and stop targets, and they are simply exec targets to the locations of the startup and shutdown scripts:

  <target name="start" description="Start Tomcat application">
    <exec executable="${catalina.home}/bin/startup.sh"/>
  </target>

and:

  <target name="stop" description="Stop Tomcat application">
    <exec executable="${catalina.home}/bin/shutdown.sh">
      <arg value="-force"/>
    </exec>
  </target>

The value of the argument to shutdown.sh is that if I define:

  CATALINA_PID="/usr/local/tomcat/bin/.catalina.pid"

then catalina.sh will save the pid in the file and then on shutdown.sh it'll do a nice kill -9 on that pid and make sure it dies. This is really important because I need to kill the Tomcat instance and I need it to die right now.

Given that we have the standard remove and install targets from the default Tomcat Ant build.xml file, then all I need to do is glue these together:

  <target name="update" description="Update the application and bounce the server">
    <antcall target="remove"/>
    <antcall target="stop"/>
    <antcall target="start"/>
    <waitfor maxwait="3" maxwaitunit="minute" checkevery="500">
      <http url="http://localhost:8080/index.html"/>
    </waitfor>
    <antcall target="install"/>
  </target>

What's happening here is that we're removing the web app from the Tomcat instance, and then shutting him down forcefully. Then we're starting him back up, but since the startup.sh is asynchronous I need to wait until I can get a page back. When I can, then I'll install the web app again.

All in all, it's pretty sweet. It's not as nice a knowing that it's smart enough to unload the shared library, but there's nothing I can do about it. Actually... now that I think about it, I think it'd be better if the loader was smart enough to see that it's the same bloody file and link into it without an issue. But I'm clearly not as clever as these guys.

What I've got is workable, and given the limitations I have (29West and Tomcat), it's as good as I can expect to do for now.

Today I've spent a lot of time adding a new way of getting data into my web app. The Shop has standardized on 29West, and I've seen it in a lot of the smaller shops, but there's got to be something more to it... cost... performance... something because Tibco is the 800lb Gorilla, but not used in a lot of the shops around town. I'm guessing it's cost. 29West may be a lot more reasonable. In any case, it's different from the majority of messaging systems I've used in the past in that there's no central messaging server. You don't connect to a server and then get a topic. Instead, you simply connect into the system and then find your topic.

It's interesting.

It's using some multicast/broadcast system to talk to the other 29West clients on the network. You connect in a machine-to-machine scheme - publisher to subscriber. It's going to be more efficient with network switches as there's no single point that all messages have to go through. Makes sense. Looks a little like magic to simply "find" the publisher, but that's good coding for you.

If there's a downside, it's in the way you have to set up the communications. It's pretty nasty. You have to create a context attributes and then populate that with configuration values, and then from there you make a context, and then a context thread to handle the socket maintenance so you don't have to do it yourself, and then you find your topic for this context, and then either a source (publisher) or receiver (subscriber) for this topic and context.

It's all needed for maximum flexibility, but really... if you're going to make it look like magic, then why not just do the context, have it take a simple map of attributes - or nothing if the defaults will do, and have it fire up a thread. Then it's find the topic and you're off to the races.

You shouldn't need to make a source or receiver... those should be done in the context, or topic. Send and receive... can't get a lot simpler than that.

I was talking to one of the 29West guys here in the Shop, and it became much clearer to me as to why it was this way: 29West is C libraries and every one of their APIs are just shells onto these C libraries. In that case, it makes sense to have a not-very-object-oriented view of things. Reasonable, and you only need to code it up one.

I haven't had a lot of experience with the performance of it, but I'm guessing I will in the coming months. For now, I'm glad I have test sources and test receivers to make sure I can move messages around - box to box, window to linux, all that. Tomorrow I'm going to have to deal with a few nagging things about the impact on Tomcat, and see if I can't get data from the guy sending it to me to me.

This morning I was reading this article about favorite prgramming fonts, and I have to agree with the list. It's really amazing that the author and I have seemingly traveled the same font path from Monaco to Panic Sans to Consolas. But he had one I hadn't seen - a variant of Consolas (it appears) called Inconsolata. He likes it over Consolas because it's free, but I have to say, in addition to that little 'plus', the biggest difference I like about it is the native spacing. It always seemed to me that Consolas had a little too much line spacing. Inconsolata solves that problem.

So have a read and see what you think. You can download Inconsolata and try it. Worth the time.

I've been working on a few pesky HTML/CSS layout issues today and finally got them worked out. If you don't specify the widths of td elements in the page, then the browser is going to attempt a rough layout on the screen and then after it's all up there (placed), it goes back and does the fine-tuning on the location to move it to it's final location. Most times it's not a big deal, but when it's little movements, it's sort of annoying. I had a few of these today.

The lesson: specify everything. Don't make the browser figure anything out. Do it all beforehand and then you won't get this secondary movement.

What I had was a table with several things in it, and since the page was largely static once it was loaded (AJAX refreshed the data) it wasn't a huge deal, but it was annoying and I wanted to make this app as smooth as possible.

I started with defining the widths of the td elements. Not bad, but still not all there. Then I did a few more. And a few more. In the end, they were all specified, but still I was having movement on one page.

Then it hit me - maybe it was the table's cellspacing and cellpadding. So I used 1 for the padding and 0 for the spacing and then took into account the padding on all horizontal calculations. Bingo!

The HTML docs say the default is 0 for both these, but that's not what Firefox and Chrome were using. With these values and carefully counting all the pixels, I get exactly what I want.

It pays to be precise with layout.

Now it's perfectly set up initially and there's no secondary move. Just what I was hoping to have.

I was adding a few things to my web app today and needed to implement a simple FIFO queue in JavaScript, but didn't need a fancy class. Thankfully, the JavaScript Array has everything I need. There's already a push() and pop() which make a LIFO queue, and if you use shift() you get the first element in the array which makes it a FIFO queue. Pretty sweet.

But the really cool part was making a queue of complex data structures. Sure, you can make JavaScript objects, and push() and shift() them, but you can make a queue that has multiple queues in it and then in your own push() method, you push the different parts onto the different queues.

Say I needed to have a queue of names and sizes. I could have the code at the top of my page:

  var  myQueue = { name: new Array(), size: new Array() };

and then later have the method:

  function push(name, size) {
    myQueue.name.push(name);
    myQueue.size.push(size);
  }

So that if we need to process what's in the queue you might say:

  function processAll() {
    while (myQueue.name.length > 0) {
      doIt(myQueue.name.shift(), myQueue.size.shift());
    }
  }

This is a really slick little way to handle data structures. This isn't real rocket science, but it's fast, simple, and for my application, it's just what I needed. Make a queue, populate it with event data, and then process them en masse.

Not bad.