Corrected Major Dark Magic Problem – Valid Dates
As I was trying to test the demand from Dark Magic in the Quantum Lead app, I noticed that I was getting a nasty parsing error - the dreaded UTF-8 encoding. Argh! I looked up what I might be able to do, and the solution seemed pretty simple. In the clojure code where we make the response from Dark Magic, we needed to make sure to set the charset in the headers:
(defn return-json [ob] {:status 200 :headers {"Content-Type" "application/json; charset=UTF-8"} :body (json/generate-string ob)})
But still, when I ran this code, I didn't get the JSON decoding error in Ruby, but I also didn't get any data. There must be something wrong! And sure, enough, there was.
No dates.
The pipeline code expected to see a start_date and end_date in the demand set to know if this data is valid for the current running date/time. If not, then it doesn't use it. I looked at the code in Dark Magic, and sure enough - we were throwing away the dates we were getting from the original source. I guess my co-worker who wrote this didn't hear me say those were important. (Hint: he did, he's just not disciplined)
So today I figured out enough clojure to change the code and get all these things into the system. That means new fields in the database, new insert methods, extract methods, comparison changes. Lots of things had to change, but it was worth it, because when I was done, it was working with the encoding and all the dates.
It started out very frustrating, but ended up being a great learning experience.