What's it all about, Alfie?

Today I got a call from the original developer of this project I've somewhat taken over. He still has components of this project that are relatively new, and I haven't even been introduced to. Needless to say, there's no documentation whatsoever - other than what I've put into the codebase as I've been working on it.

So today when he called and said that one of his reports was missing a lot of data, I knew it was not going to be fun. He was really busy with this other project that his desk was depending on, and it was going to be up to me to figure out what had happened and how to get it back to "right".

Documentation: Zero

As I've said, this guy didn't document a thing. There's a few high-level 10,000 ft. documents about how things are supposed to work. Problem is, this project was built with Spring and while it's certainly got benefits, I have to say making a reasonable, understandable, documented system isn't one of them. Confused? Sure. Exotic XML config files? Roger.

I don't mean to harp on this, but I'm going to because it's just that important. If you can describe the system at a high level as simple and flexible, then the implementation ought to be simple and flexible. It seems that Java has created this entire cadre of developers that over-design even the simplest of things just because they can. It's one of the most frustrating things I've run into in years.

Every language has it's detractors, and my favorites are no exception. I'm not one to throw stones, living in a glass house, but this really does seem to be something of a pandemic. Systems built with tools that almost restrict the developer from making useful, well-documented, code and easily understood code.

Unit Tests: Useless

This is the project that has the most incomplete, yet copious, set of unit tests that I've ever seen. The unit tests passed completely, but at the same time, they didn't point out why I might have absolutely no data in the report. Does that sound like a good set of unit tests? Yet, I spent the better part of two days updating these same unit tests when I added a feature that took me about an hour. What's right with that?

If you're going to have unit tests, and more than that, have integration "unit" tests, then don't you think it should cover the case where a great deal of the result is empty? If it's a coding mistake, it should be caught in the tests. If it's a data issue that would blow out that much data (like a null pointer), then again, it should be flagged as at least a warning.

No tests failed or even logged problems.

Getting to the Bottom of It All

I looked at the SVN change logs for all the files. Nothing there would make all the data disappear. I looked at the data in the database - maybe a lead there, but no, it was a false alarm. I looked at the data moving around the system at a level that wasn't easy to locate as there wasn't any documentation (as I've said), but in the end I was able to see the data coming out of the test system and the production system. Interestingly, the data at this level wasn't blank. It appeared that the Flex client was interpreting the data as something it didn't want to display.

Now we were getting somewhere.

When I looked at the data more closely I noticed that the numbers were remarkably similar. Interesting. Then I noticed that one part of the data was the wrong symbol. Very interesting, now. So I asked the original developer on this point, and he said that the symbol I had was wrong. OK, let's go there.

I had put the configuration of the system into a database so that it's far easier to maintain and basically is driven off the data that's maintained by the individual desks. When I put in these normalizing contracts I put in what I believed to be the right contracts. Turns out, for this report, for this group of instruments, it was wrong.

A one field change in the database and a restart of the Tomcat instance and Bingo! data was there in the report.

Lessons Learned

At the end of the experience I was sad to realize that I hadn't really learned a lot from this other than what I already knew: poor documentation, poor tests, poor checking on the client, the list goes on and on. While I know it's like whipping a dead horse, as long as I'm forced to work with this system it'll continue to be a source of anguish to me. It's got a lot of nice things in it, but the design and documentation aren't among them.

The code is consistent for the most part, and the goals and attempts are admirable, but in the end it's another victim of that pandemic of over-design and under-documentation. Make it simple... make it clean... work on the documentation... make it something that when another developer looks at it, it makes them code better. Set the Gold Standard for code. Be better than you have to be.

I learned those lessons years ago, and I only home some of them rub off.

Wonderful is the Twitter update!

Today I got a tweet from the OmniGuys saying that OmniGraphSketcher has gone Final 1.0. Great news, with a raft of changes in this cut the application really is ready to go. I'm glad I got in on this as I do love visualizations and graphs, in particular.

Had to get it.

This morning I saw that the Mozilla crew finally released Firefox 3.5! This is great news as I have been waiting on this for a long time for work. I'm hoping to see the performance of Firefox 3.5 on Windows to be up to snuff and capable of handling my web pages without crashing. The initial results seem to indicate that it's possible that this version of Firefox is far better at memory usage than the previous versions - and even better than Google Chrome! That would be great.

Even if that's not the case, it's nice to see Firefox making a big move in the JavaScript area. Very much appreciated.

Apple has released a big update to iPhoto '09 (8.0.4) that is supposed to fix a serious crashing bug that effected some users. I didn't experience it, but then again, I just got my iPhone to start uploading those pictures! Had to get it as I'm planning on doing a lot more picture taking.

Today has been a really stressful day. I was getting hit with production problems at the same time I was learning that the code changes I'd made in this project I inherited weren't sufficient to stand-alone, and in order to really work I had to make significant changes to the code.

Very stressful. Didn't eat until late. Gulped it down.

Had to fix the code first, it had the greater risk of me not getting it done. The problem was that I believed that the changes I had made completely removed the necessity for the config files (Spring - ick) to have this list of instruments. Instead, I wanted to read them from the database where they were held. Makes tons of sense to not have to maintain the same data in two forms. Problem was, when I removed the files I found that they had actually been referenced in other places, and I needed to get those lists out of these other places.

That didn't turn out to be as easy as I'd have hoped. With the components responsible for their own loading, it was difficult to see where I might be able to use Spring to reference the containers with their database-loaded information. But there was. Then I hit another problem - a circular reference in the Spring.

This was getting to the point I was ready to re-write the whole thing.

But I got an idea - get a 'core' set of containers and have them queried for their instruments as opposed to doing everyone. Now we're talking. With that, I was able to fix things up in a few hours. Still... the unit tests were the vast majority of the time, and a significant hassle as the containers are not easy to make in isolation.

Once I got all that done I was able to check it in and send off an email to the other developer on the project and tell him to run through all the tests to make sure that I haven't broken anything. Just making sure.

Then I could focus on the production problem... that was another toughie, but I was glad to be able to find the difference. It wasn't so much a problem as it was a difference. The production box was using specific 'Ask' prices and the users were thinking it should be using 'Bid' prices - and as you can imagine that would change things considerably.

When I matched those conditions the numbers started lining up very nicely. So I asked them if it was going to be 'Ask' or 'Bid', and they said 'Bid', so I changed production to use 'Bid' and we should be good tomorrow.

Finally, I had another production problem with the data from London. It turned out that one of the London machines had crashed on me - it happens about twice a day in London. All the other sites are fine, so I'm convinced it's the VMs in London. I passed on the information to the London guys and they said they'd be looking into the VMs tomorrow. Good enough.

Super heavy, stressful day, but dog gone it, I got it all done. All of it. That's pretty nice. Now I can go home and feel I earned my dollar today.

Well, I've had my iPhone for a few days now, and I have to say they were 100% on the money. It's a fantastic little device. The keyboard is great - I have no problems typing on it, and the auto-correct helps me when I do make a mistake. The speed of the 3GS is great, and the ability to use WiFi and 3G is really nice.

Of course, it's the price that's blowing me away today. I expected to pay a lot more for the service, and so in comparison, it seems very reasonably priced. Love it.

I've picked up a few apps, but nothing major yet. I think I need to get a nice SSH client, but we'll see how that pans out with the apps being certified now for iPhone OS 3.0 - maybe there will be something new coming out that'll blow my socks off. There are a few updates I'd like to see - specifically WordPress for iPhone, but I can wait for that.

I great little device. Amazing. Now I just need to find a clip/holder so it'll fit on my backpack...

I spent a lot of today still working on getting the unit tests fixed up for the changes I made. I've said enough on this already, but the amazing thing to me was when I got through them, I was able to add in the significant new functionality in about 30 mins. This goes back to the best way I work - create human test cases, and understand the code to know what needs to be tested in each release cycle. It can't always be done, that's true, but when I can work that way, I get a lot more done.

I'm hoping to get to do a lot more of the latter and a lot less of the massive unit testing.

Well... I took the plunge and I'm so glad I did! This evening we went to the Orland Park Apple Store and picked up the 32GB black iPhone 3GS. Amazing. The thing that really helped clinch the decision was the fact that Joseph's phone had an accident and his charger was no longer able to charge his phone. It was in the phone, not the charger, and when we went to the Verizon store, they said it was out of warranty, and yet he was available for an upgrade.

Rather than go for the upgrade, Liza and I agreed to look at the AT&T store and start the process of moving the family from Verizon to AT&T. The reason for this is simple - better service coverage. Liza goes to Jewel and Target, and when she's in these stores, she can't get any signal from the CDMA Verzion service. But her best friend, Kelly, on AT&T is able to get GSM service everywhere in these stores. It's a simple matter of frequency, wavelength and a Farady Cage.

So we moved over Joseph, and started the 700 minutes Family Plan with unlimited texting. Then, when I went to the Apple Store, it was simply a matter of adding a line ($9.95/mo) and then the data plan for the iPhone ($30/mo) and that was it! This was about $70 less than I thought I'd have to spend. What an incredible deal!

So I got it home, took about an hour to move all my stuff from my old iPod Touch and then it was ready to go! I took pictures of Liza and the kids so I'd see their pictures when they called - and sent emails. I had a blast with it. Got a few apps that I'd wanted to have once I had a camera, and even a game or two. It was fun.

So I just need to set things up at work with the second charger so I can keep it charged, and I'll be ready to go. Very nice.

I've been spending all day trying to get the unit tests for this inherited application working, and I'm closer to being done, but I'm not done. It's like the rock Prometheus has to roll up the hill, only to see it slide back down. If I want to add in new features to the codebase, then I have to face these tests, and it takes me far longer to update the tests than it does to test the code, or write it in the first place.

I know there are paradigms that say "Write the Tests First", but I can't actually imagine doing that unless you're writing small, easily testable, modules. This integration testing is not for unit tests, and I keep harping on it, as I'm continually forced to deal with them.

In order to make things work today, I had to create "Mock" services that just fed in the test data. It's again making no real progress in the testing code, we don't know that the database interface is working, and while we can run those tests, it's not doing the kind of integration testing that is going to make me feel any better.

So it's slugging it out all day with these things. I really do get very tired of it.

I've been looking at Eclipse on Windows for a bit, because I have to for this one project I'm working on at work, and thinking about what it can do made me think that I've probably been a little stuffy about the IDE. I'm probably not giving it the credit it deserves. Refactoring, automatic compilation... it's probably a nice tool, and I should give it a try.

So I'm downloading the new 'Galileo' version (3.5) from the Eclipse Download site, and going to replace the older version I have. I'm not sure if it's a big difference, but it's probably something I should give a go - if for no other reason than at least trying to understand what the developers are aiming for.

Better to understand and have a working knowledge of something than to dismiss it out of hand.