What's it all about, Alfie?

I was working this afternoon and happened to catch out of the corner of my eye the founding partner I work for talking to someone (whom I didn't recognize, but had a suit so he's not "native"). He was very animated... hands up in the air, big smile on his face, looking my way.

I took out my earbuds and looked over at him with a look that said "Is there anything I need to do?" His response was a fantastic compliment: "I was just bragging about you".

Wow.

Double Wow.

I've always hoped that the things I did were noticed, and here I've been at the job a little less than 4 months and already it's something they are that happy with. That's really neat. I mean - Holy Cow! You don't have to do that stuff, and certainly not tell me about it.

Sweet. Put a big smile on my face.

Well, after a little false start, BBEdit 9.2.1 os out with a handful of bug fixes. Some of these must have been nasty for this kind of release, but maybe the BareBones guys are starting to have smaller, more frequent, bug releases. Might be nice, but their stuff is pretty solid to begin with.

No matter... had to get the update.

They have done a lot with OmniGraphSketcher to get it to 1.0 RC 1 status. I have to say that I enjoy the tweets from the development group as it makes sure I'm kept up to date of any happenings the team has to share. This update is pretty extensive with improvements in speed, import data handling and general UI issues. It's getting better and better.

Very nice to see.

In the last few weeks I've written several times about my dislike of extensive unit testing when it comes to large, complex systems. Unit tests are perfect for building blocks - simple model objects that have well-defined inputs and outputs that can be extensively tested. It's my days in the IC fab industry that makes me wary of testing. The guys at HP had amazingly hard test vectors to test every component as quickly as possible for all possible defects. It wasn't easy.

But I've been working on a system where the unit tests are large, complex, and difficult to maintain -- and don't test enough of the system. But it's assumed that these tests are complete coverage of the system. Well... today it bit me in the behind, and I have solid reasons why I now hate this philosophy.

This morning, I found that the update from last evening wasn't working exactly as it should. In fact, it wasn't working properly for a lot of cases. I was trying to see if is was the data - hard to tell, that's for sure. Maybe it was the install... nope. I was getting pretty desperate. Then someone mentioned checking the Tomcat localhost log. Bingo!

There were a bunch of exceptions that told me exactly what the problem was. Why is this not in the catalina.out log? I'm guessing it's the way this webapp is set up, but all my exceptions should go to catalina.out. Here nor there at this point, I had found the problem.

The problem was that one component was sending a record of six elements, and the other system was expecting seven. Clearly, this was the problem, and when I fixed the sender to add the missing data, it all worked fine. But it was 3 hours of a nightmare because of the impact to production.

All the unit tests would never have picked this up because it was between two components in the same system. More over, there was no testing of this specific feature in my tests, the other developer's tests, even Q/A didn't test this. While I'm not mad at anyone (other than myself), I think this highlights the folly of thinking that unit testing of large, complex systems, is anything more than a waste of time. Small, easily identifiable blocks - sure. But webapps? Please.

So now I've learned something valuable, and when the next person tries to spout the Gospel According to Unit Testing, I'll have a good story to share with them.

Camino 1.6.8 is out today with a few fixes that are certainly needed, but with the latest news of Firefox 3.5 and what's happening with the other browsers, I wonder how long Camino will stay an active project. It's already showing signs of lagging - very much like Firefox 3.0 did.

Only time will tell.

This morning, Adium told me that 1.3.5 was out and it fixes connection issues with Facebook, Yahoo, and MSN, as well as updating the version of libpurple in use. It's a nice upgrade, but I haven't really noticed any serious connection issues with these guys, but maybe I'm just lucky.

Anyway, the bet IM client on the Mac... gotta love that.

I have been working in an inherited codebase that uses the Google Collections library to extend the Collections in Java 5. I have to admit, they have done the set of classes that I'd have expected from Google - fits in with the existing objects, but adds those classes that can really be quite handy from, say, C++ and STL.

Multimap is one, missing in the JDK, but picked up by the Google Collections and implemented pretty nicely. They have the standard interface and then several implementations, each with it's particular plusses and minuses. I look at this very much in the same vein as the visualization library that I've been using for a while - very nice, but clearly done for something they had planned.

Not bad. I may spend a little more time looking into the library now that I've had to find it. Could be a nice addition like the visualization library.

I was reading one of my sites this morning and ran across this study of memory footprint usage by Chrome 3.0, Safari 4.0, Opera 10 and Firefox 3.5. I have to say that I'm impressed with Firefox 3.5's numbers. I have been using Chrome 2.0 for a while and very happy with it in terms of minimal memory footprint for my web application, but if the new Firefox 3.5 is going to best it, that will be even better.

I'll have to see what happens in real-life tests, but it's really encouraging to see this kind of progress in the Firefox line. It's been sitting on 3.0 for a while as others have jumped ahead. It'd be great to see Firefox retake the lead for a while.

I noticed this morning that Xcode 3.1.3 is out from Apple. I'm sure this is the release that supports the iPhone OS 3.0 SDK, and for those of us that haven't yet gotten that SDK, this probably has just a few little enhancements. No matter, I'm a big fan of Xcode and I'd be getting the update anyway. Great tools.

UPDATE: I'm having a slight problem downloading the new docs within the Help system for Xcode 3.1.3. It's saying the password isn't right, but I know it is as I was able to download the other docs update within Help just fine. I'll keep trying every so often, and if it fails, I'll download them manually from the Apple Developer site.

Today I spent the majority of the day updating jUnit tests for this new feature that I need to add to this system I have inherited. The addition of the feature wasn't too bad. There were wrinkles as we're not really handling the booking of positions like you might in an accounting system with all the levels, etc. that you might otherwise do, but that's part of this business - it's a limited subset of products and how they are organized. No biggie, still it took a little bit to work it in.

The vast majority of the day was the updating of the jUnit tests. I've thought about this a lot, and there's a lot to be said for Unit Tests, and there are real benefits to Integration Tests, but I think that confusing one for the other can not only make for bad testing, but a false sense of security.

First, it's important that Unit Tests include - at a minimum, the following:

• Edge Conditions - if you're not checking for the minimum and maximum allowable values then you're not really testing the unit. If you use only one datapoint, the "black box" might just work for that point and nothing else.
• Improper Inputs - if you code up checks on the inputs, then you need to test those so that you're certain that should the user pass you in an illegal value, you'll catch it.

this is by no means a comprehensive list - it's just the minimum you need to test in order to have a reasonably high level of confidence that what you've written is working. Face it... it's all about code-coverage. If your tests only hit 10% of the code in a module, the remaining 90% can have millions of bugs. You need to try and get as good code coverage in your tests in order to increase your confidence in the tests.

Then there's the issue of using unite testing frameworks to do integration testing. This is a really an extension of the problems associated with incomplete code coverage testing. Face it, if you're not testing all of your units completely, then trying to test three or more with the same coverage is going to make things even more dicey. Sure, you can do it, but the tests become less and less about really creating assurance as they are about habit.

Someone used to writing unit tests may not be able to draw the line. They're so used to writing them they just can't stop. They drive them far past their point of usefulness. Oh sure... there is some utility in the limited tests, but it's nothing like an assurance that things are working properly. It's more like "Yeah, I haven't messed this horribly."

This is really nothing more than simple testing with live data.

I have seen cases where the unit testing is complete and amazing. But in those cases, it's a lot of work and maintaining the tests is a significant bit of work. I've also seen cases where the testing doesn't ensure nearly anything, but it's still a ton of work because of the integration set-up required.

Test appropriately. This includes unit tests and human-driven tests.