{"id":2180,"date":"2009-06-23T16:00:09","date_gmt":"2009-06-23T21:00:09","guid":{"rendered":"http:\/\/bobbeaty.com\/wp\/?p=2180"},"modified":"2009-06-24T04:00:33","modified_gmt":"2009-06-24T09:00:33","slug":"on-the-false-security-of-unit-testing-large-systems","status":"publish","type":"post","link":"https:\/\/bobbeaty.com\/wp\/archives\/2180","title":{"rendered":"On the False Security of Unit Testing Large Systems"},"content":{"rendered":"<p><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/bobbeaty.com\/wp\/wp-content\/uploads\/2007\/09\/cubelifeview.gif\" alt=\"cubeLifeView.gif\" border=\"0\" width=\"128\" height=\"128\" align=\"right\" \/><\/p>\n<p>In the last few weeks I've written several times about my dislike of <em>extensive<\/em> unit testing when it comes to large, complex systems. Unit tests are <em>perfect<\/em> for building blocks - simple model objects that have well-defined inputs and outputs that can be <strong><em>extensively<\/em><\/strong> tested. It's my days in the IC fab industry that makes me wary of testing. The guys at HP had amazingly hard test vectors to test every component as quickly as possible for all possible defects. It wasn't easy.<\/p>\n<p>But I've been working on a system where the unit tests are large, complex, and difficult to maintain -- <strong><em>and don't test enough of the system.<\/em><\/strong> But it's <em>assumed<\/em> that these tests are <em>complete coverage<\/em> of the system. Well... today it bit me in the behind, and I have solid reasons why I now <strong>hate<\/strong> this philosophy.<\/p>\n<p>This morning, I found that the update from last evening wasn't working exactly as it should. In fact, it wasn't working properly for a lot of cases. I was trying to see if is was the data - hard to tell, that's for sure. Maybe it was the install... nope. I was getting pretty desperate. Then someone mentioned checking the Tomcat <strong>localhost<\/strong> log. Bingo!<\/p>\n<p>There were a bunch of exceptions that told me <strong>exactly<\/strong> what the problem was. Why is this not in the <tt>catalina.out<\/tt> log? I'm guessing it's the way this webapp is set up, but all my exceptions <strong>should<\/strong> go to <tt>catalina.out<\/tt>. Here nor there at this point, I had found the problem.<\/p>\n<p>The problem was that one component was sending a record of six elements, and the other system was expecting <strong>seven<\/strong>. Clearly, this was the problem, and when I fixed the sender to add the missing data, it all worked fine. But it was 3 hours of a nightmare because of the impact to production.<\/p>\n<p>All the unit tests would <strong>never<\/strong> have picked this up because it was between two components in the same system. More over, there was no testing of this specific feature in my tests, the other developer's tests, even Q\/A didn't test this. While I'm not mad at anyone (other than myself), I think this highlights the folly of thinking that unit testing of large, complex systems, is anything more than a waste of time. Small, easily identifiable blocks - sure. But webapps? Please.<\/p>\n<p>So now I've learned something valuable, and when the next person tries to spout the Gospel According to Unit Testing, I'll have a good story to share with them.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In the last few weeks I&#8217;ve written several times about my dislike of extensive unit testing when it comes to large, complex systems. Unit tests are perfect for building blocks &#8211; simple model objects that have well-defined inputs and outputs that can be extensively tested. It&#8217;s my days in the IC fab industry that makes [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,6],"tags":[],"class_list":["post-2180","post","type-post","status-publish","format-standard","hentry","category-coding","category-cube-life"],"_links":{"self":[{"href":"https:\/\/bobbeaty.com\/wp\/wp-json\/wp\/v2\/posts\/2180","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/bobbeaty.com\/wp\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/bobbeaty.com\/wp\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/bobbeaty.com\/wp\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/bobbeaty.com\/wp\/wp-json\/wp\/v2\/comments?post=2180"}],"version-history":[{"count":3,"href":"https:\/\/bobbeaty.com\/wp\/wp-json\/wp\/v2\/posts\/2180\/revisions"}],"predecessor-version":[{"id":2183,"href":"https:\/\/bobbeaty.com\/wp\/wp-json\/wp\/v2\/posts\/2180\/revisions\/2183"}],"wp:attachment":[{"href":"https:\/\/bobbeaty.com\/wp\/wp-json\/wp\/v2\/media?parent=2180"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/bobbeaty.com\/wp\/wp-json\/wp\/v2\/categories?post=2180"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/bobbeaty.com\/wp\/wp-json\/wp\/v2\/tags?post=2180"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}