Today I spent the majority of the day updating jUnit tests for this new feature that I need to add to this system I have inherited. The addition of the feature wasn't too bad. There were wrinkles as we're not really handling the booking of positions like you might in an accounting system with all the levels, etc. that you might otherwise do, but that's part of this business - it's a limited subset of products and how they are organized. No biggie, still it took a little bit to work it in.
The vast majority of the day was the updating of the jUnit tests. I've thought about this a lot, and there's a lot to be said for Unit Tests, and there are real benefits to Integration Tests, but I think that confusing one for the other can not only make for bad testing, but a false sense of security.
First, it's important that Unit Tests include - at a minimum, the following:
- Edge Conditions - if you're not checking for the minimum and maximum allowable values then you're not really testing the unit. If you use only one datapoint, the "black box" might just work for that point and nothing else.
- Improper Inputs - if you code up checks on the inputs, then you need to test those so that you're certain that should the user pass you in an illegal value, you'll catch it.
this is by no means a comprehensive list - it's just the minimum you need to test in order to have a reasonably high level of confidence that what you've written is working. Face it... it's all about code-coverage. If your tests only hit 10% of the code in a module, the remaining 90% can have millions of bugs. You need to try and get as good code coverage in your tests in order to increase your confidence in the tests.
Then there's the issue of using unite testing frameworks to do integration testing. This is a really an extension of the problems associated with incomplete code coverage testing. Face it, if you're not testing all of your units completely, then trying to test three or more with the same coverage is going to make things even more dicey. Sure, you can do it, but the tests become less and less about really creating assurance as they are about habit.
Someone used to writing unit tests may not be able to draw the line. They're so used to writing them they just can't stop. They drive them far past their point of usefulness. Oh sure... there is some utility in the limited tests, but it's nothing like an assurance that things are working properly. It's more like "Yeah, I haven't messed this horribly."
This is really nothing more than simple testing with live data.
I have seen cases where the unit testing is complete and amazing. But in those cases, it's a lot of work and maintaining the tests is a significant bit of work. I've also seen cases where the testing doesn't ensure nearly anything, but it's still a ton of work because of the integration set-up required.
Test appropriately. This includes unit tests and human-driven tests.
