What's it all about, Alfie?

I was just looking at my Dropbox usage, and realized that the menu drop-down had actual thumbnails of the images that were uploaded to Dropbox! What a slick trick! I'm sure this is not new, but it's new to me, and while I don't have a huge amount of data on Dropbox, I have been using it for a long time, and when the free space runs out, I will certainly be a paying customer. It's just an amazing tool, and it's been integrated with a lot of really nice apps - like Glui and VoodooPad.

It's when apps add features that sneak up on me that I'm really happy. Getting a new app, you kind of expect it to have the functionality you're getting. But to have an existing app add nice features is really a little bit of joy.

Very nice, Dropbox! Very nice.

While it's no GitHub, I've been working with Bitbucket for a few projects with a friend, and I have to say it's not bad at all. I happen to like the helpful hints GitHub offers on making a new repo, but the help Bitbucket has is very fair, and reasonable. It's got a little different UI, which is fine, and it works pretty much the same so that you don't have to come up to speed with a new workflow if you're switching back and forth from Bitbucket to GitHub.

I looked at Bitbucket a long time ago, and it wasn't nearly as similar to GitHub as it is now, and that makes a lot of sense. It's got Pull Requests, and it's got a very nice source view:

Many will say It's just like GitHub's - which is a good thing. Face it - you copy what's a great design and works well. This is one of those cases.

The thing I like about Bitbucket is the free repos for small teams - up to 5 people. In short, they have a different approach than GitHub - repos are free, people are charged. It's a nice approach because it means that non-Open Source projects that you work on with a friend can be private without a cost. GitHub charges for the first private repo, and I don't begrudge them that - I'm a paying GitHub user. But I like the different approach because it gives small groups a leg up.

I'm not going to stop using GitHub, but it's nice to see that Bitbucket exists and seems every bit as good as GitHub for the task. It's something I'll remember.

Well... I just pushed the commits that added the web server to the CryptoQuip repo with my friend. I have been able to get the speed down to that of the Obj-C code, and it was really interesting to me where the hold-up was. I needed to add some logging with timings so that I could get performance numbers for changes as I ran them over and over. But what I found was that the real bottleneck was in the possible? predicate function that was being used to filter the possible words to those that matched the pattern of each cypherword.

What I needed to do was to switch to pmap to make it efficient on getting the matches:

  (defn possible?
    "Function to see if the cyphertext and plaintext have the same pattern of
    characters such that they could possibly match - given the right legend."
    [ct pt]
    (if (and (string? ct) (string? pt) (= (count ct) (count pt)))
      (let [ctc (count (distinct ct))
            ptc (count (distinct pt))]
        (if (= ctc ptc)
          (= (count (distinct (map str ct pt))) ctc)
          false))
      false))

where I now look at the (distinct) length of the cypher and plaintext before making the map of pairs and counting them. This saves a good bit, but the biggie was in breaking out the creation of the possibles sequence:

  (defn match-up
    "Function to create a sequence of all the cypherwords and the possible
    matches to it in a way that we can easily time this for performance
    tuning."
    [quip]
    (let [qw (vec (distinct (.split quip " ")))
          go (fn [cw] (let [poss (filter #(possible? cw %) (get words (count cw)))]
                        { :cyphertext cw
                          :possibles poss
                          :hits (count poss) }))]
      (sort-by :hits < (pmap go qw))))
 
  (log-execution-time! match-up
    {:msg-fn (fn [ret q] (format "%s words filtered" (count ret)))})

Here, the biggie is splitting up the matching using pmap to make sure that we work on this matching problem as fast as possible. It's all still very functional, as the source lists are immutable, and it's a simple filtering problem.

With this, we're in the 65 msec range for the solution, and that's about where the ARC-based Obj-C code was, and that's pretty nice.

This morning I saw that I got an offer letter from one of the shops I have been interviewing with and it was pretty much exactly what I'd heard from the CTO last night on a phone call. I have to say that I'm pretty impressed with the speed of getting the letter out. I told him there was no rush, but they got it done right away. Nice touch.

I've sent an email to another of the companies I talked to to inform them that I'd gotten the letter. Just to let them know that I'd need to know within a few days if they were interested. Just a courtesy email. I don't expect anything from them, but it's what Ive been told is polite. And I want to be polite. They have all been very nice in this interview cycle.

It looks like I'll have a restful Christmas break and then get to the new job at the first of the year. New chapter in life and new job. Not bad.

This morning I finished up a different implementation of the CryptoQuip solver that I've been migrating from Obj-C to ruby - and now to clojure. A friend has already done a port of the logic, but he made slightly different implementation scheme, and therefore a different execution profile. This is a lot more along the lines of the the Obj-C and ruby ports, just in clojure.

What really impressed me what the exceptionally tight implementation of the functions in clojure. For the most part, the code is about the size of the ruby code, except where it comes to the reentrant attack code. There, clojure is once again, just amazing. The ruby looks like this:

  # This is the recursive entry point for attempting the "Word Block" attack
  # on the cyphertext starting at the 'index'th word in the quip. The idea is
  # that we start with the provided legend, and then for each plaintext word
  # in the 'index'ed cypherword that matches the legend, we add those keys not
  # in the legend, but supplied by the plaintext to the legend, and then try
  # the next cypherword in the same manner.
  #
  # If this attack results in a successful decoding of the cyphertext, this
  # method will return true, otherwise, it will return false.
  def do_word_block_attack(idx, key)
    p = @pieces[idx]
    cw = p.cypherword
    p.possibles.each do |pt|
      have_solution = false;
      if cw.can_match?(pt, key)
        # good! Now let's see if we are all done with all the words
        if (idx == @pieces.count - 1)
          # make sure we can really decode the last word
          if key.incorporate_mapping(cw.cyphertext, pt)
            # if it's good, add the solution to the list
            if (dec = key.decode(@cyphertext))
              @solutions << dec
              have_solution = true
            end
          end
        else
          # OK, we had a match but we have more cypherwords
          # to check. So, copy the legend, add in the assumed
          # values from the plaintext, and move to the next
          next_key = key.clone
          if next_key.incorporate_mapping(cw.cyphertext, pt)
            have_solution = do_word_block_attack(idx + 1, next_key)
          end
        end
      end
      # if we have any solutions - stop
      break if have_solution
    end
    @solutions.count > 0
  end
end

and the clojure code looks like:

  (defn attack
    "Function to do the block attack on the quip, broken down into a sequence
    of the cyphertext words, and their lists of possible plaintext words, as
    well as the index in that sequence to attack, and the clue (legend) to use.
    This will return the first match to the attack and that's it."
    [quip pieces idx clue]
    (if (and (coll? pieces) (map? clue))
      (let [{cw :cyphertext poss :possibles} (nth pieces idx)
            last? (= idx (dec (count pieces)))]
        (some identity (for [pt poss
                             :when (matches? clue cw pt)
                             :let [nc (merge-clue clue cw pt)]
                             :when nc]
                         (if last?
                           (decode nc quip)
                           (attack quip pieces (inc idx) nc)))))))

which is just amazingly simple and easy to follow.

The execution times don't beat those that my friend's code, but maybe it's due to loading, or maybe he'll be able to find something in the performance I missed. But you just can't beat the simplicity of the clojure code.

Just amazing!

Today my daughter and I went to BestBuy to get a few things - and among them was the Sonos system for the house. I've been looking at this for a long time, and even gone to the store on at least three occasions to get them, but then walked out of the store without a thing. I just didn't see it as a need. Then I saw her putting together her IKEA furniture listening to some tunes from her phone. I knew this was pretty standard for my kids, but still... it was just a little silly when we can make the house be filled with the sound from the same phone.

So we got them. She's got a Play:1 for her room, and a Play:3 for the living area downstairs. I got a Play:5 for the living room upstairs, and a Play:3 for the kitchen, and finally a Play:1 for the office. She gets to control all the sound downstairs, and I can control it upstairs. It's really amazingly impressive.

The sound fills the house, and as you walk around from room to room the sound appears to already be there - well... because it is! There's no "walking away" from the music, and it's not too loud in any room - as you can control the speakers independently. This is just exactly what I was hoping for.

The set-up could not have been easier. Set up the base unit - a "Boost" unit - connected to the router, and then just go around plugging in one speaker at a time, and hitting a few buttons on the iPhone (or Android) app and on the speaker to register each one, and then it's all done. I was a little worried that this would require something on my Mac, but not so - just an iPhone will do nicely.

Now I am able to stream music and it's honestly a little better than the AirPlay from Apple, as I've had drop-outs there, but have experienced nothing like that with Sonos. Very cool. And it's just in time to be playing all my Christmas favorites!

As I didn't have a lot to do this afternoon, I decided to check on the state of X11 on OS X 10.10.1 - and it's actually not bad at all. I was able to get Quartz 2.7.7 from the download site, and it installed easily enough. All the normal things run, and it's a complete X11 we just need the apps - or in the case of SSH access, getting to a linux machine for the X11 apps on the remote host.

Because I got rid of my home machines with X11 a while back, I'm not sure how much this is going to get used any more, but it's nice to know that it works before I really need it, so that I know I can go to it in a time of need.

I've been using Dash for a while now. It started out with a really odd looking cat as the icon, and it was a little iffy in the beginning, but it got better - a lot better. Today, it's the best documentation viewer I have ever used. It's just stunning. And it's fast. I can't believe how fast it is, but maybe it caches everything in a docket in memory - who knows? But Wow... it's easy to use, expansive on all the docs it covers, and the speed and ease of use are simply hard to beat.

I have a set for Clojure coding which includes clojure, postgres, and Java, and I have one set for ruby - with ruby, postgres and Java (for JRuby). All told, it's amazing how access to the right docs immediately is so powerful.

I had a great phone call from an old friend last night. He's known me since the second grade. That's over 45 years. I don't have another friend that's even close to that long, and he's been a wonderful friend. He was remarking last night that he felt I was making progress, and that he was glad about that - and he should know... more than a decade as a nurse on the psych ward puts him in a position to know about recovery. So it was nice to hear that from him.

This morning I was looking at a pull request from one of the younger - and quite smart - developers in the group. I looked at what he'd done, and it was all just very sloppy. Basing the pull request off a deleted branch used in another pull request, and then not checking dependencies in the libraries, and then just changing things because he wanted to.

Typical kinds of things I've seen from a lot of smart, but not very disciplined, developers. They are used to their intelligence covering for their lack of attention to details and general discipline. Most of the time it works, but that's really just luck more than anything else. They aren't really compensating for it, they're just obscuring it. It's still there - the lack of detail, etc. and it'll come back to haunt the group and the code.

But I really can't say anything to him, because until he wants to change, he's not going to be receptive to any other person telling him he needs to change. We are all like that. I'm on a path now that I would have never voluntarily taken... in fact, I fought not to take it - but it had to be. Sooner or later, I had to realize that my marriage wasn't what I needed, nor was it what Liza needed, and maybe we shouldn't have been married, or maybe things just changed. Whatever the reason, it had to happen.

So it will be for the smart-but-lazy developer.

And at the same time, I've been told by many friends that the environment I've been in for months is not good for me. Again, I have been fighting that this should be a good fit, but that's based on a wish - not a fact. As I've interviewed with other shops, I've realized that I'm not a guy to be "just" a developer. That's going to be too big a mis-match, as I'm going to have decades of experience over my bosses, and that's only going to lead to issues.

Here again, I didn't want to see it - I just didn't want it to be true. But it was. I need to stop pretending that I'm happiest as just a developer. I'm a creative person, and consequently, I want to spend the majority of my time creating, but I can't shut off 30 years of professional programming, and all the mistakes made, and lessons learned. And when I am asked to make one of those mistakes again by a manager that hasn't yet made it, I really have to do it. But I can't. I just can't.

So I really need to be in the kinds of positions that I'm looking at currently - Senior Tech positions... Leads, Architects, decision maker. Something where my experience is a requirement so that it's not overlooked, but leveraged. I was again not thrilled with the idea of this a few months ago when I turned down promotions here at The Shop, but I see that was no different than me not wanting my marriage to end - wishful thinking, but sadly not accurate.

Yet I'm learning. Slowly, yes, but I'm learning. Accept. Forgive. Move on.

So I'm honestly looking forward to some of these opportunities I've interviewed for. There's a small company that needs someone to just help out. Lots of things to do, and because there aren't all that many people, there's lots to do. Then there's another place that's looking for real-time trading experience, and that's always a ton of fun to build - and the people there are smart and experienced so I'm not dealing with the 20-somethings that play cards during lunch and come in at 9:15.

Finally, there's one that might be the most interesting of all - as a tech lead for a line of business working side-by-side with the head of the business to make sure that the systems will scale, be robust, and keep moving in the right direction. This would require me to use everything I've learned and apply it on a daily basis - creatively and experientially.

Yet I know it's a path. I don't know that I'll ever have a day where I don't hurt about the marriage and family that might have been. Or the job or group that might have been. But those are just stories, and not reality. But the pain is very real.

I was a big fan of The Muppet Show when I was younger... I never missed a Sunday night of that show. Such incredible memories... "Pigs in Space"... The Electric Mayhem... and of course - my favorite: Beaker. I could go on and on about why this character but it all comes down to the fact that he wanted so badly to belong, but at the same time he was always the victim - even if it was all accidental.

I don't know if this was the same for others, but I have to believe there was a genius in casting when they came up with Beaker and Dr. Bunsen Honeydew. But over and over again Henson came up with the perfect characters for each of us to identify with.

Anyway... listening to the Muppets Christmas album made me laugh at Beaker.