What's it all about, Alfie?

We wanted to be able to log, to the database, the name of the function that was called and there was no obvious way to do that. There were some half-baked ideas, but we really needed to have a little more robust scheme. So we made this:

  (defn fn-name
   "Function to get the name of the function passed in from the basic
   information of the function - or in the case of a hooked function, the
   metadata placed on that function by the hook."
   [f]
   (-> (or (:robert.hooke/original (meta f)) f)
       (class)
       (str)
       (cs/replace #"^class " "")
       (cs/replace #"\$" "/")
       (cs/replace #"_" "-")))

where:

  [clojure.string :as cs]

is in the namespace's require function.

This will take the normal function - or one that Robert Hooke has captured for additional magic, and return it to the caller.

Very handy thing to have for logging... very.

Bret and I finally got the CORS support for ring middleware working in our applications. It required that we get the starting point for the CORS support from another project, and then augment it to make sure it was working for all the cases we needed.

Basically, you need to start with:

  [rwilson/ring-cors "0.1.9"]

in your project.clj, and then in the server.clj for the library, we reference it:

  [ring.middleware.cors :as cors]

and then create the middleware function:

  (def wrap-cors
    "This is a simple convenience definition for enabling the CORS support in
    middleware for a service. The use of this will be far simpler than including
    all this in every service."
    #(cors/wrap-cors % :access-control-allow-origin #".+"
                       :access-control-allow-headers ["Origin" "Content-Type"
                                                      "Accept" "Authorization"
                                                      "Last-Modified" "Credentials"
                                                      "X-Request-Id" "X-Session-Id"]
                       :access-control-allow-methods [:get :put :post :delete]))

And with this, you can then simply put this into your middleware stack and it takes care of all the work:

  (def app
    "The actual ring handler that is run -- this is the routes above
     wrapped in various middlewares."
    (-> app-routes
        wrap-json-with-padding
        (wrap-ssl-redirect { :ssl-port (cfg/jetty :https-redirect) })
        handler/site
        wrap-cors
        wrap-params
        wrap-cookies
        wrap-logging
        wrap-proxy-loggly
        wrap-gzip))

We have a shared library that has a lot of really useful functions in it for all the apps we build, and one of the really useful things is to round to a fixed number of decimal places. Now, this isn't Rocket Science, but it is nice to have written well once, and then not have to mess with ever again.

The trick is that you need it to be fast for all Clojure datatypes as well. And several of the more common implementations aren't very fast due to boxing and the overhead associated with that. So we made some pretty simple little functions:

  (defn to-2dp
    "Function to simply resolve a number to a decimal (double), and then round
    it to 2 DP and return it. Pretty simple, but very useful."
    [x]
    (/ (math/round (* 100.0 (parse-double x))) 100.0))
 
  (defn to-3dp
    "Function to simply resolve a number to a decimal (double), and then round
    it to 3 DP and return it. Pretty simple, but very useful."
    [x]
    (/ (math/round (* 1000.0 (parse-double x))) 1000.0))

but we had a problem. Some of the inputs gave us a lot of grief because of some nasty rounding. And it wasn't all values - it was just those values that had a hard time being expressed in the floating point format.

The solution was to accept some of the inefficiency of rat eJVM BigDecimal and the advanced representation it had, and use that:

  (defn to-2dp
    "Function to simply resolve a number to a decimal (double), and then round
    it to 2 DP and return it. Pretty simple, but very useful."
    [x]
    (/ (math/round (* 100.0 (bigdec (parse-double x)))) 100.0))
 
  (defn to-3dp
    "Function to simply resolve a number to a decimal (double), and then round
    it to 3 DP and return it. Pretty simple, but very useful."
    [x]
    (/ (math/round (* 1000.0 (bigdec (parse-double x)))) 1000.0))

and then things settled down.

This morning I was dealing with yet another change to some clojure functions that are in a library that we have created at The Shop to make it easy to make the services that we make, and easy for new clojure developers to wrap their heads around. It's not perfect, or ideal, but it does strike a nice balance between purely functional code and re-use and state in the library.

Face it - connection pooling is state. It needs to exist, but it's not what you'd like to have if you could avoid it. So you bury it in a library, so that the new clojure developers can use this library with the connection pools clearly hidden off screen doing their thing.

It's a compromise. But one that favors the new clojure developer. And most days, that's the folks I have to work with. They typically know Java, or C#, but it's a rare individual that comes into the shop with a lot of clojure experience, and usually they "get it" about the decisions, and it's something they pretty easily adapt to.

But then there are others.

Today I've been dealing with Steve. Steve is a C# developer that hates his job. He's hated it for a long time. He hates the language he's using (C#). He hates the tools he is having to learn to do his job. He pretty much hates everything about his job.

But he wants to work in clojure.

Yet he's not really interested in learning the landscape. He wants to change it to be how he thinks is should be. I don't blame him - he questions a lot because he trusts no one. Having asked him this direct question, and receiving the affirmative response, I know beyond a shadow of a doubt, this is exactly who he is. But to not learn what is before suggesting what should be is a little short-sighted.

That he's annoying about it is just "icing on the cake" for me.

So this morning I decided that emotion has no place in this conversation. Neither does his opinions. I have listened, I have tried to explain, I have listened more, and tried to show why this is the path we have taken. All of this is immaterial to him, and so in the end, I have to make a decision, list the reasons for this decision, and then move on.

I don't relish being a jerk. Or a dictator. But at this point, more conversation is wasting time. It's time to accept that this is the paradigm we are working under, and that's it.

I do not expect him to like it.

One of the things that I find I need to do every now and then is to have a case-insensitive regex pattern match in clojure, and I always have to spend 15 mins looking for it on the net. Not that it's hard to find, but it's 15 mins, and if I know I wrote about it here, then I can do a much quicker search, and get the same answer.

The key is that the Java regex parser recognizes two interesting options to the source for the regex: (?i) and (?iu). The first, (?i), is the flag to indicate that the entire regex is to be considered case-insensitive. The second, (?iu) is for unicode support.

So how's it work?

  (re-find #"Cook" "This is how to cook.")
  => nil
 
  (re-find #"(?i)Cook" "This is how to cook.")
  => "cook"

It's simple and easy, and typically saves you from calling lcase on the string before the regex test.

Over the past several months we have built several Clojure/Javascript apps at The Shop, and each one has needed to have some level of authentication, and thankfully, that's all been provided by the Security Service that was already built and maintained by another guy at The Shop. It works, what else would we need? You get authentication tokens (UUIDs) and you can get information about the user based on the token, and you can't really guess an auth token for a user, so they are pretty decently secure.

But they were app-specific because we tied them to the cookies on the pages, and those pages were all from a site that was different for each app we were building. This meant that you'd have to login to three, four apps a day with the same credentials, and still not have a clean, guaranteed way of invoking a page in one app from a page in another.

This is what I wanted to solve.

This morning, it came to me - there had to be a way to store the auth tokens in a cookie that was across the apps, and then just use that as a standard for all apps. Simple. Since it was still secure (enough), we didn't have to worry about folks yanking the token, or even knowing who the token belonged to.

The question was How?

Thankfully, jQuery has this all solved with the domain: tag in the $.cookie() function. All we needed to do was to have a consistent upper-level domain for the apps that also worked for local development.

What I came up with was a simple function:

  function loginDomain() {
    var here = document.domain;
    if (here === 'localhost') {
      return here;
    }
    var parts = here.split('.');
    parts.shift();
    return parts.join('.');
  }

so that we would have a single login for the production hosts on theshop.com and another for the development hosts on da-shop.com. This would work perfectly if I could update all the places that the cookie was accessed.

What I learned was that the accessing of the cookie did not need to have the domain: tag, but setting it - or deleting it did. So for example, to read the token, I'd say:

  var tok = $.cookie('shop-auth-token');
  if (tok) {
    // ...
  }

and no matter where the cookie was saved, if it's in the "domain path", then it's OK. But when I needed to set the cookie I'd need to:

  var exp = new Date();
  exp.setTime(exp.getTime() + (8 * 60  * 60 * 1000));
  $.cookie('shop-auth-token', tok, { domain: loginDomain(),
                                     expires: exp );

and when I needed to invalidate it:

  $.removeCookie('shop-auth-token', { domain: loginDomain() });

Once I got these all set up, and in each application, one login on the specific login domain would then be successfully used on all apps in that domain with this code. Sweet! It worked like a charm, and it took me about 90 mins to figure this out, and put it in all the apps we have.

Big win!

Postgres is an amazing database. With 9.4's JSONB datatype, and a little clojure code, I can save data structures to and from Postgres, and have them query-able by any JDBC client. It's just amazing. But recently we had a problem at The Shop where we needed to index a field in a JSONB element, but it was a date, and so we ran into the problem of Immutable Functions.

Simply trying to create an index on the field:

  CREATE INDEX idx_d ON my_table ((fields->>'DateStarted'))::DATE;

didn't work with Postgres saying you can't have a mutable function in the creation of the index. So it was time to dig out the browser and look up what's going on. Sure enough, this was something well-known, and all involving the conversion of dates by locale, and why that was not allowed for an index.

Makes sense, but I still needed an index because the queries using this field in the WHERE clause were taking 6+ min, and we needed to really bring them down - like way down. Then I found that you could make functions that were tagged as IMMUTABLE and so made these:

  CREATE OR REPLACE FUNCTION mk_date(src VARCHAR)
  RETURNS DATE
  immutable AS $body$
    SELECT src::DATE;
  $body$ LANGUAGE SQL;
 
  CREATE OR REPLACE FUNCTION mk_datetime(src VARCHAR)
  RETURNS TIMESTAMP
  immutable AS $body$
    SELECT src::TIMESTAMP;
  $body$ LANGUAGE SQL;

the reason I chose to use ::date and ::timestamp is that they handle NULL values very well, and the corresponding functions don't do that so well.

With these functions, I was then able to create the index:

  CREATE INDEX idx_d ON my_table (mk_date((fields->>'DateStarted')));

and then as long as I used the same functions in my WHERE clause, everything was great.

The result was a 1000x increase in speed - 319,000 msec to 317 msec. Amazing.

I do love Postgres.

There are days that I don't know how I'll make it through the day back to my little house, and the one place that I really feel safe. And I'm continually surprised that every day around 6:00-ish, I find myself looking at that facade, and unlocking the door, and walking in and feeling an amazing sense of relief. I'm not better - I'm just a little bit stronger. I can take the heartache and pain that seems to follow my day-to-day existence.

A very nice friend of mine once said "It either gets better, or you get used to it. Either way, it seems like it's better." Not bad advice. It's not something that's easy to heed, but it is true nonetheless. Very true, indeed.

So this past month has been a lot more more of getting used to it than it getting really better. Just gotta keep slogging through the muck...

The Shop is big into Datadog, and it's not a bad metrics collection tool, which I've used very successfully from clojure. Based on the use of the local Datadog Agent (freely available from Datadog) and how easily it's placed on linux hosts - AWS or otherwise, it's a clear win for collecting metrics from your code and shipping them to a nice graphing/alerting platform like Datadog.

The code I've set up for this is pretty simple, and based on the com.codahale.metrics java libraries. With a simple inclusion into your project.clj file:

  [io.dropwizard.metrics/metrics-core "3.1.0"]
  [org.coursera/dropwizard-metrics-datadog "1.0.2"]

you can then write a very nice metrics namespace:

  (ns ns-toolkit.metrics
    "This is the code that handles the metrics and events through the Dropwizard
    Metrics core library, which, in turn, will ship it over UDP to the DataDog
    Agent running on localhost."
    (:require [clojure.tools.logging :refer [infof debugf warnf errorf]])
    (:import [com.codahale.metrics MetricRegistry]
             [org.coursera.metrics.datadog DatadogReporter]
             [org.coursera.metrics.datadog.transport UdpTransportFactory
                                                     UdpTransport]
             [java.util.concurrent TimeUnit]))
 
  ;; Create a simple MetricRegistry - but make it only when it's needed
  (defonce def-registry
    (delay
      (let [reg (MetricRegistry.)
            udp (.build (UdpTransportFactory.))
            rpt (-> (DatadogReporter/forRegistry reg)
                  (.withTransport udp)
                  (.withHost "localhost")
                  (.convertDurationsTo TimeUnit/MILLISECONDS)
                  (.convertRatesTo TimeUnit/SECONDS)
                  (.build))]
        (.start rpt 5 TimeUnit/SECONDS)
        reg)))
 
  ;; Somewhat faking java.jdbc's original *connection* behavior so that
  ;; we don't have to pass one around.
  (def ^:dynamic *registry* nil)
 
  (defn registry
    "Function to return either the externally provided MetricRegistry, or the
    default one that's constructed when it's needed, above. This allows the user
    the flexibility to live with the default - or make one just for their needs."
    []
    (or *registry* @def-registry))

And then we can define the simple instrumentation types from this:

  ;;
  ;; Functions to create/locate the different Metrics instruments available
  ;;
 
  (defn meter
    "Function to return a Meter for the registry with the provided tag
    (a String)."
    [tag]
    (if (string? tag)
      (.meter (registry) tag)))
 
  (defn counter
    "Function to return a Counter for the registry with the provided tag
    (a String)."
    [tag]
    (if (string? tag)
      (.counter (registry) tag)))
 
  (defn histogram
    "Function to return a Histogram for the registry with the provided tag
    (a String)."
    [tag]
    (if (string? tag)
      (.histogram (registry) tag)))
 
  (defn timer
    "Function to return a Timer for the registry with the provided tag
    (a String)."
    [tag]
    (if (string? tag)
      (.timer (registry) tag)))

These can then be held in maps or used for any reason at all. They automatically send their data to the local Datadog Agent over UDP so there's no delay to the logger, and since it's on the same box, the likelihood that something will be dropped is very small. It's a wonderful scheme.

But one of the things that's not covered in these metrics is the Gauge. And there's a really good reason for that - the Gauge for Datadog is something that is read from the Datadog Agent, and so has to be held onto by the code so that subsequent calls can be made against it for it's value.

In it's simplest form, the Gauge is just a value that's read by the agent on some interval and sent to the Datadog service. This callback functionality is done with a simple anonymous inner class in Java, but that's hard to do in clojure - or is it?

With Clojure 1.6, we have something that makes this quite easy - reify. If we simply add an import:

  (:import [com.codahale.metrics Gauge])

and then we can write the code to create an instance of Gauge with a custom getValue() method where we can put any clojure code in there we want. Like:

  ;;
  ;; Java functions for the Metrics library (DataDog) so that we can
  ;; constantly monitor the breakdown of the active docs in the system
  ;; by these functions.
  ;;
  (defn cnt-status
    "Function that takes a status value and finds the count of loans
    in the `laggy-counts` response that has that status. This is used
    in all the metrics findings - as it's the exact same code - just
    different status values."
    [s]
    (reify
      Gauge
      (getValue [this]
        (let [sm (first (filter #(= s (:status %)) (laggy-counts)))]
          (parse-int (:count sm))))))
 
  (defn register-breakdown
    "Function to register all the breakdowns of the loan status counts
    with the local Datadog agent to be sent to Datadog for plotting. This
    is a little interesting because Datadog will call *these* functions
    as needed to get the data to send, and we will control the load by
    using memoized functions."
    []
    (.register (met/registry)
      "trident.loan_breakdown.unset"
      (cnt-status nil))
    (.register (met/registry)
      "trident.loan_breakdown.submit_to_agent"
      (cnt-status "Submit to Agent"))
    (.register (met/registry)
      "trident.loan_breakdown.submit_to_lender"
      (cnt-status "Submit to Lender"))
    (.register (met/registry)
      "trident.loan_breakdown.submit_to_lender_approved"
      (cnt-status "Submit to Lender - Agent Approved"))
    (.register (met/registry)
      "trident.loan_breakdown.lender_approved"
      (cnt-status "Lender Approved")))

What I like about this is that I can allow the Datadog Agent to hit this code as often as it wants, and don't have to worry about the freshness of the data - or an excessive loan on the server resources for being hit too much. I can simply memoize the functions I'm using and then control the load on my end. It's very clean, and very nice.

I was asked recently by my manager to look into code coverage tools for clojure because some of the other Senior Devs at The Shop wanted to have code (test) coverage numbers automatically generated for all projects as a part of continuous integration (TeamCity), and then available for all to see and track. I can certainly understand what they are trying to achieve - testable code that allows them to feel comfortable changing the code after the original author is long gone.

With complete test coverage, the theory goes, you can make a change, and then run these tests and prove to yourself that you haven't broken something because you didn't take the time to really understand the codebase.

It's an understanable goal from a management perspective... but even there, I have to think that this has never worked in my experience - and there's no way to have really foolproof tests. Yet today I read a nice post, which contained a quote from Rich H. about testing:

A bad design with a complete test suite is still a bad design. -- Rich H.

And the author, Alex, also verbalizes a lot of the concerns I've had over the years about tests. I can remember adding a feature in 15 mins and then spending several hours updating the unit tests because there were so many that had double-coverage, and yet none could (politically) be removed. Tests, it seems, are a lot like gold bricks - once people get some, they are very reluctant to get rid of any.

Yet they are code, and cost to maintain just like code. You can't think that tests are "free" once they are written... at least not if you're honest with yourself. And to Rich's point, it's better to think first and then attack the problem than it is to believe that tests are everything you need.

It was a very illuminating article. I'm glad a have a link to it now. I'll be sending it to a lot of folks who talk about tests in the future.