Starting to Use CouchDB
The decision was made late last week that we really should try to use some document database - like CouchDB for saving our run state for metrics extraction and historical analysis. We had initially planned on using MySQL, as it's the dominant database in The Shop, but that was going to require that we flatten the ruby objects to some set of key/value pairs, or come up with some ORM that would properly store and load things. Neither was really super attractive, and so we had Jeff in the group take a look at CouchDB. I knew MongoDB wasn't the answer, because I'd used it previously, but there were a few nice sounding things CouchDB had that could really tip the scales in it's favor.
Most notably were the views. These are basically the same things that you'd expect in a SQL database, but in the document sense they can be arbitrary map/reduce schemes implemented in javascript, and stored on the CouchDB. This means that we can make some interesting views for the metrics that gather data across different documents and make it presentable in a very simple and efficient way.
I'm thinking that generating the majority of the metrics are possible in this way, and then the stuffing these values into a visualization system shouldn't be too bad. We'll have to see as I'm still in the very early stages of using this, but it certainly has some interesting potential for what we're trying to do.