Logging all Incomplete Processing
This morning I decided that it'd be nice to have a complete list of all the merchants that didn't successfully complete their processing. Since we are now processing everything - a recent change to the code to make sure that we know exactly every single merchant is getting completely processed, we can now look at each merchant and make sure that they got through the critical processing phases. If it didn't "pick up" the right data, then we can assume that it didn't get to that point. The point of this is that we can then be sure that every merchant completed processing.
I log this, and write them to CouchDB, so we can keep a complete record of all the issues, and then updated the summary script to list the number of incompletely processed merchants so we can watch them over time.
Nice. This is starting to really get close to verification that all was done as it was supposed to have been done.