Clojure Makes a Great Reporting Framework
This morning I had to really smile... and even do my Happy Dance in my chair. I used something I found in the clojure library adworj to make reporting on Facebook ad server data as nice and clean as the code I lifted from adworj. I really liked adworj, but he wanted to handle credentials one way, and I had to have them in a separate store, so there were enough differences that the only really good things were the metadata on the reports from AdWords.
So what I decided to start with was a slight variant on the report metadata that he used. I started with a similar metadata structure, but instead of the macros and defrecord, I chose a simpler map of the data:
(def reportstats-fields "The complete list of all allowed fields in the Facebook ReportStats reporting system keyed on the nice clojure name, and including the Facebook name and an optional parser for the returned data." {:time-start {:name "time_start" :parse parse-long} :time-stop {:name "time_stop" :parse parse-long} :date-start {:name "date_start" :parse to-date-time} :date-stop {:name "date_stop" :parse to-date-time} :account-currency "account_currency" :account-id {:name "account_id" :parse parse-long} :account-name "account_name" :adgroup-id {:name "adgroup_id" :parse parse-long} :adgroup-name "adgroup_name"})
where I've clearly truncated the list of fields from Facebook, but you get the idea. There is either a string that's the Facebook field name, or there's a map with two keys: :name for the Facebook name of the field, and :parse for the function to mars the value into it's clojure data element.
My real divergence starts with how the reverse-mapping is done. I start by having a defined set of all the field names, as opposed to computing it on the fly over and over again:
(def all-fields "Set of all valid keyword/field names for the Facebook ReportStats reports." (set (keys reportstats-fields)))
and then another that defines a map for how to get a Facebook field into closure-land:
(def coercions "Map of the Facebook field names to the `:parse` functions for those fields - if they exist in the report definition. If they don't, then don't map anything and they won't then be handled in the reading/parsing." (into {} (for [[n md] reportstats-fields :let [nf (if (string? md) md (:name md)) cf (if (string? md) identity (:parse md))]] [(keyword nf) (fn [v] [n (cf v)])])))
What I like about this approach is that we don't have to deal with the overhead of doing this for each report each time it's run. While that's probably not a big overhead, why? Why spend any time on this once it's done? The report structures are fixed in both cases, and there's just no reason for that level of flexibility.
What the coercions map gives me is a collection of functions to map the values coming back from Facebook - keyed by their Facebook field name, and suitable for inclusion into a map - with the correct clojure keyword for the field name. For instance:
(defn coerce-record "Function to take a record from Facebook, and apply the known mappings to get it **back** into decent clojure names and datatypes." [m] (into {} (for [[k v] m] ((get coercions k) v))))
This function simply creates the right map (record) from the Facebook map (record) in a very simple, easy way. If there's a change in the formatting, just change the structure and it'll automatically be fixed in subsequent calls. That's nice.
Need to have a new field? Add it. Or change the name... it's all so easy to report like this.
Best of all, the real reporting code is just a vector of keywords to define what to get, and then a simple all. Very nice indeed!