Well, after a little false start, BBEdit 9.2.1 os out with a handful of bug fixes. Some of these must have been nasty for this kind of release, but maybe the BareBones guys are starting to have smaller, more frequent, bug releases. Might be nice, but their stuff is pretty solid to begin with.
No matter... had to get the update.
In the last few weeks I've written several times about my dislike of extensive unit testing when it comes to large, complex systems. Unit tests are perfect for building blocks - simple model objects that have well-defined inputs and outputs that can be extensively tested. It's my days in the IC fab industry that makes me wary of testing. The guys at HP had amazingly hard test vectors to test every component as quickly as possible for all possible defects. It wasn't easy.
But I've been working on a system where the unit tests are large, complex, and difficult to maintain -- and don't test enough of the system. But it's assumed that these tests are complete coverage of the system. Well... today it bit me in the behind, and I have solid reasons why I now hate this philosophy.
This morning, I found that the update from last evening wasn't working exactly as it should. In fact, it wasn't working properly for a lot of cases. I was trying to see if is was the data - hard to tell, that's for sure. Maybe it was the install... nope. I was getting pretty desperate. Then someone mentioned checking the Tomcat localhost log. Bingo!
There were a bunch of exceptions that told me exactly what the problem was. Why is this not in the catalina.out log? I'm guessing it's the way this webapp is set up, but all my exceptions should go to catalina.out. Here nor there at this point, I had found the problem.
The problem was that one component was sending a record of six elements, and the other system was expecting seven. Clearly, this was the problem, and when I fixed the sender to add the missing data, it all worked fine. But it was 3 hours of a nightmare because of the impact to production.
All the unit tests would never have picked this up because it was between two components in the same system. More over, there was no testing of this specific feature in my tests, the other developer's tests, even Q/A didn't test this. While I'm not mad at anyone (other than myself), I think this highlights the folly of thinking that unit testing of large, complex systems, is anything more than a waste of time. Small, easily identifiable blocks - sure. But webapps? Please.
So now I've learned something valuable, and when the next person tries to spout the Gospel According to Unit Testing, I'll have a good story to share with them.
I have been working in an inherited codebase that uses the Google Collections library to extend the Collections in Java 5. I have to admit, they have done the set of classes that I'd have expected from Google - fits in with the existing objects, but adds those classes that can really be quite handy from, say, C++ and STL.
Multimap is one, missing in the JDK, but picked up by the Google Collections and implemented pretty nicely. They have the standard interface and then several implementations, each with it's particular plusses and minuses. I look at this very much in the same vein as the visualization library that I've been using for a while - very nice, but clearly done for something they had planned.
Not bad. I may spend a little more time looking into the library now that I've had to find it. Could be a nice addition like the visualization library.
I was reading one of my sites this morning and ran across this study of memory footprint usage by Chrome 3.0, Safari 4.0, Opera 10 and Firefox 3.5. I have to say that I'm impressed with Firefox 3.5's numbers. I have been using Chrome 2.0 for a while and very happy with it in terms of minimal memory footprint for my web application, but if the new Firefox 3.5 is going to best it, that will be even better.
I'll have to see what happens in real-life tests, but it's really encouraging to see this kind of progress in the Firefox line. It's been sitting on 3.0 for a while as others have jumped ahead. It'd be great to see Firefox retake the lead for a while.
I noticed this morning that Xcode 3.1.3 is out from Apple. I'm sure this is the release that supports the iPhone OS 3.0 SDK, and for those of us that haven't yet gotten that SDK, this probably has just a few little enhancements. No matter, I'm a big fan of Xcode and I'd be getting the update anyway. Great tools.
UPDATE: I'm having a slight problem downloading the new docs within the Help system for Xcode 3.1.3. It's saying the password isn't right, but I know it is as I was able to download the other docs update within Help just fine. I'll keep trying every so often, and if it fails, I'll download them manually from the Apple Developer site.
Today I spent the majority of the day updating jUnit tests for this new feature that I need to add to this system I have inherited. The addition of the feature wasn't too bad. There were wrinkles as we're not really handling the booking of positions like you might in an accounting system with all the levels, etc. that you might otherwise do, but that's part of this business - it's a limited subset of products and how they are organized. No biggie, still it took a little bit to work it in.
The vast majority of the day was the updating of the jUnit tests. I've thought about this a lot, and there's a lot to be said for Unit Tests, and there are real benefits to Integration Tests, but I think that confusing one for the other can not only make for bad testing, but a false sense of security.
First, it's important that Unit Tests include - at a minimum, the following:
this is by no means a comprehensive list - it's just the minimum you need to test in order to have a reasonably high level of confidence that what you've written is working. Face it... it's all about code-coverage. If your tests only hit 10% of the code in a module, the remaining 90% can have millions of bugs. You need to try and get as good code coverage in your tests in order to increase your confidence in the tests.
Then there's the issue of using unite testing frameworks to do integration testing. This is a really an extension of the problems associated with incomplete code coverage testing. Face it, if you're not testing all of your units completely, then trying to test three or more with the same coverage is going to make things even more dicey. Sure, you can do it, but the tests become less and less about really creating assurance as they are about habit.
Someone used to writing unit tests may not be able to draw the line. They're so used to writing them they just can't stop. They drive them far past their point of usefulness. Oh sure... there is some utility in the limited tests, but it's nothing like an assurance that things are working properly. It's more like "Yeah, I haven't messed this horribly."
This is really nothing more than simple testing with live data.
I have seen cases where the unit testing is complete and amazing. But in those cases, it's a lot of work and maintaining the tests is a significant bit of work. I've also seen cases where the testing doesn't ensure nearly anything, but it's still a ton of work because of the integration set-up required.
Test appropriately. This includes unit tests and human-driven tests.
I was reading Gruber today and he pointed to an article about a new free font - Anonymous Pro. It's very clean and nice, and at a 10pt Bold, it's perfect for BBEdit. It's dark enough to be very readable, but not too smashed to be unreadable. It's tight, has wonderful 'greater than' and 'less than' signs... it's just perfect for BBEdit.
I tried it in Xcode, but I have to say here, the rendering they use in Xcode is best served by Panic Sans. But still, it's not bad.
I tried it in MacVim, but again, Inconsolata was better here for how they rendered text. Still, not bad.
So it's going to depend on what app and what you want to see, but for a light background on BBEdit, this is amazing.
Today I spent a good bit of time planning out the changes I need to make in this one project I'm taking over. It's not the way I'd build it, but that's really beside the point... it is and I have to work with that. Plain and simple.
So I'm looking at why a strike for an option that should be 1.34 is really 1.34000003674 - I've run into this before in my previous job: rounding in the format change from string to float or double. It's just the way things are. So I needed to see how things were being converted and then see where I might be able to put in code to properly round to something like 1.0e-4, or something.
I found the data in the database was stored as a real, and that's not a good sign right off the bat, as I think that's going to be mapped into a Float by JDBC as opposed to the Double that might have prevented this. But the problems didn't end there. The code I found mapped the value to a String and then later, it was mapped to a Double and then used. By then, I'm not at all surprised that it had the extra digits. Too many conversions. Way too many.
But again, in order to minimize the impact to the code, it was smarter to look at where the options were being created and insert the rounding on the strike at that point. It's not what I'd have wanted to do, but again, I had something that was close to right, and so it wasn't really worth the risk of changing all the mapping and database code when the problem could be addressed very simply in the setter code of the new option:
public synchronized void setStrike(Double value) { _strike = Math.rint(value * 1.0e4)/1.0e4; }
with this, we'll get the strike rounded to the nearest 1/100th of a cent. Easy to do, and it'll wipe out all the "problem digits" that we're seeing.
But I wasn't done yet. There was more code that needed fixing on the strikes. There was a place in the code where they were trying to create the "short-hand name" of the option and it had the following code to display the strike:
double decimal = strike - Math.floor(strike); int strikeInt = new Double(strike).intValue(); ... if (decimal > 0.) shortName.append(strike); else shortName.append(strikeInt);
where the goal is to render the Double 1.5 as "1.5" and the Double 95.0 as "95". Basically, if there's no fractional part, then don't show the decimal point. Seems good, but this is a horrible way to do it. A far better way would be to say:
DecimalFormat fmt = new DecimalFormat("0.####"); ... shortName.append(fmt.format(strike));
where the formatter will properly round to four decimal places and also drop the entire decimal part if there isn't one to display.
Sure, the DecimalFormat isn't thread-safe, but I'm not expecting it to be. Make one, use it, drop it. They are meant to be very lightweight classes - which is why they aren't thread-safe. But in this case, they are perfect for the occasion.
Then I got into another problem that I needed to tackle and realized that I didn't see where the original author had made any allowances for the feature. I looked more and still found nothing. I then shot off an email as I dug even further. By the time I talked to him, I was convinced that he simply didn't have any modeling of this particular business rule in the code. It was all based on configuration files that duplicated these business relationships he didn't want to include.
I talked to him and he fessed up about not making any allowances for this because he thought it was wrong. I pointed out that all he did was to push it into static config files in the server, and he agreed.
Right or wrong, a business rule is a business rule. It's not about "correctness". It's about how the users think of things. If they choose to think of things as "colors" then model the colors - just as they think of them. Don't make the quality judgement on the way things are - just use them.
I had done this with his code - from the get-go, but he wasn't as accommodating of the users. I told him what I needed to do and why, and was very nice about the judgement, but in the end, we need to add in this concept as it's critical to being able to do several things that are globally important, even if they aren't important to him.
It's important to understand that there's a time to correct something because its wrong, and leaving it as-is is going to cause more problems in the future, and when it's time to let it flow over you and work with the imperfect system that you've been given.
I've been using the Google Visualization AnnotatedTimeLine for quite a while. It's a pretty nice tool, and it's got the backing of Google, for all that means. Good and Bad. This morning I was pointed to the Chronoscope web site where they seem to have built something on top of the GWT that can replace the AnnotatedTimeLine pretty much completely. It even has a 'compatibility' skin where it looks like the AnnotatedTimeLine (or so the docs say).
I'm always on the lookout for something that's better, faster, and cleaner than the stuff I'm using. This certainly has a lot of things to like. Right now, my biggest concern for the AnnotatedTimeLine is the fact that it 'flashes' on all updates. It's not fast but part of that is the fact that I have to give it the entire dataset at each update. It'd be great if the dataset was able to be augmented, and in doing that, it only redrew the parts that changed.
I'm not sure that Chronoscope is the answer. It's no faster than the AnnotatedTimeLine for similar dataset sizes, but it's something to keep and eye on. Maybe it'll post some specs on speed and size and give me more reason to investigate it further.
They have updated Java again, once again, it's an across-the-board update:
Java for Mac OS X 10.5 Update 4 delivers improved reliability, security, and compatibility for Java SE 6, J2SE 5.0 and J2SE 1.4.2 on Mac OS X 10.5.7 and later.
This release updates Java SE 6 to version 1.6.0_13, J2SE 5.0 to version 1.5.0_19, and J2SE 1.4.2 to 1.4.2_21.
So we're at pretty up-to-date versions of Java for OS X. Not bad. It's no surprise that we're lagging behind the Sun JDKs - Apple has never really pushed the envelope on JDK releases. But it's nice to be up to date just in case.