What's it all about, Alfie?

One thing that I didn't yet have a handle on with LightTable is how to get this to act as my REPL - for an existing project. Simple one-file projects, or tests were easy, but I wanted to have a REPL that was also an editor - and I was clueless how to get that working.

So I decided to dig in and figure it out.

Turns out, it's really pretty easy. First, you have to start up LightTable, and then open up the command list and search for 'connect'. What you're going to do is to create a new connection. So find the 'connect bar':

then click on 'Add Connection':

and then select the 'Clojure' item:

and then select the project.clj file and you're in business!

Simply open up an InstaREPL, and you're in a REPL in that project. Very nice!

One of the things that I've really missed in the latest version of Mac OS X is the utility to set the default version of the JDK to use. Well, with all the clojure coding I've been doing, I have typically stuck to JDK 1.7, but recently, the auto-updater from Oracle has them shipping JDK 1.8. Yet this doesn't effect the command-line utilities. This is a pain in the neck.

So first, I got JDK 1.7.0_75 on all my machines - it's going to be the last version of JDK 1.7 that I'll get as it's off the auto-updater. Then I was sure that everything was OK, I also got the complete JDK 1.8.0_40 - which is what the auto-updater downloaded for the JRE, but not the JDK. Yes, annoying.

At that point, I started googling about what to do. Turns out, JAVA_HOME is all I need to worry about. The java, jar, java apps in /usr/bin are all respectful of JAVA_HOME. Then I found this function that I added to my ~/.bashrc:

#
# Clever trick to leverage the /usr/bin/java commands to take advantage
# of the JAVA_HOME environment variable and the /usr/libexec/java_home
# executable to change the JDK on-the-fly. This is so easy I'm amazed.
#
function removeFromPath() {
    export PATH=$(echo $PATH | sed -E -e "s;:$1;;" -e "s;$1:?;;")
}
 
function setjdk() {
    if [ $# -ne 0 ]; then
        removeFromPath '/System/Library/Frameworks/JavaVM.framework/Home/bin'
        if [ -n "${JAVA_HOME+x}" ]; then
            removeFromPath $JAVA_HOME
        fi
        export JAVA_HOME=`/usr/libexec/java_home -v $@`
#        export PATH=$JAVA_HOME/bin:$PATH
    fi
}
setjdk 1.7

At this point, I saw that the key command was /usr/libexec/java_home, and it had all the answeres I needed. I didn't need to update my PATH - just JAVA_HOME. I also could verify that I had all the versions of Java I needed:

  $ /usr/libexec/java_home -V
Matching Java Virtual Machines (7):
  1.8.0_40, x86_64:  "Java SE 8"  /Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/
                                   Contents/Home
  1.7.0_75, x86_64:  "Java SE 7"  /Library/Java/JavaVirtualMachines/jdk1.7.0_75.jdk/
                                   Contents/Home
  1.7.0_51, x86_64:  "Java SE 7"  /Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/
                                   Contents/Home
  1.7.0_45, x86_64:  "Java SE 7"  /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/
                                   Contents/Home
  1.7.0_13, x86_64:  "Java SE 7"  /Library/Java/JavaVirtualMachines/jdk1.7.0_13.jdk/
                                   Contents/Home
  1.6.0_65-b14-466.1, x86_64: "Java SE 6"  /System/Library/Java/JavaVirtualMachines/
                                            1.6.0.jdk/Contents/Home
  1.6.0_65-b14-466.1, i386:   "Java SE 6"  /System/Library/Java/JavaVirtualMachines/
                                            1.6.0.jdk/Contents/Home

And then with a simple command - in the shell of my choice - I can set - and reset the JDK I use. It's very click:

  $ setjdk 1.7
  $ javac -version
  javac 1.7.0_75
  $ setjdk 1.8
  $ javac -version
  javac 1.8.0_40

Everything else works like a charm. This is a handy little tool to have, and I'm sure I'm going to use it often as I migrate from JDK 1.7 to 1.8.

I've often said that Command and Control is a valid, useful, form of management. So is Consensus. Now the latter is a lot harder to achieve because you have to have very self-aware people in the organization. They have to be aware of their limitations, and know when it's best for the group for them to stop back, and to step up. Yet it is, without doubt, the most effective team.

But a lot can get done with Command and Control - as long as everyone does their part. The commanders have to assume responsibility for their commands, and they have to be available to make those decisions at any time - based on the needs of their subordinates to have issues resolved. After all, it's no good to have a great commander if he's never available to make the command decisions.

I've been thinking a lot about management lately, and I'd like to think I've made enough mistakes to know that I work pretty well in either form, but I think everyone works better in the consensus format - assuming they have the right teammates. After all - a good team doesn't have to be staffed will all-stars, it just has to be staffed with the right people, and then set the expectations accordingly.

What I have seen fail over and over is the organization that claims to be Consensus - but is, in fact, Command and Control. They want to appear that they trust the workers, that they have hired the right mix of people, but in the end, they really don't trust them because either they know the people aren't right (but are possibly hoping they will become the right people), or they just don't trust others.

This latter truth is really what I've seen more of than anything else - mistrust. A very smart guy - capable of being an amazing individual contributor gets promoted out of that spot, into a place where they are no longer outstanding. It's the Peter Principle all the time:

managers rise to the level of their incompetence

It's so true it almost makes me sad, reading the Wikipedia page. I can see in so many people that they have really risen out of the position where they were outstanding into a job that they really are very marginal at. They might have thought they could do it, or could learn to do it, or would get better at it, but in the end, their realization that they really are no good at their job is something that creates a great deal of friction for them - internally.

Unhappy people aren't necessarily the first to see it in themselves. They get irritable, cranky, intolerant, micromanaging - all things that the wouldn't normally want to be doing, but their current mental state is being heavily influenced by the fact that they are mad at themselves for taking this job over staying where they were and feeling a lot of job satisfaction out of doing something really well.

I don't think there's any real solution to this problem. The wikipedia page gives a few suggestions, but they are all virtually impossible to implement as human beings are involved, and the willingness to admit defeat is not something most people want to admit to. So we suffer through. Adults aren't going to change their beliefs and habits without a significant Life-Altering Event - and there's no way to predict those - nor would I want to inflict one on anyone. But that's what it'll take for these folks to re-asses the way they work.

Sadly, it's just considered normal.

I've got a few projects on Bitbucket, and while it's nice to have an alternative to GitHub's policy on private repos, the Markdown rendering on Bitbucket is really sad. The problem is simple: If I have a repo with images in it, and I want to include them in the README.md, then for GitHub I need to include something like:

  <center>
    <img src="docs/img/logo.png" width="450" height="200"
    border="0"/>
  </center>

and it'll work - but including the HTML, as per the Markdown standard, on Bitbucket is not allowed. They have written that they don't allow any HTML in their Markdown due to safety reasons. But then why does GitHub? Is it just because Bitbucket isn't using as good a parser?

In addition to the lack of HTML, the relative links in the Markdown aren't supported, either. This means that in order to include an image in the README.md, I have to do something like:

  ![Using the correct G-REPL URL](https://bitbucket.org/applieddl/
    pocket/raw/master/doc/img/grepl-url.png)

this means that the links don't work in local preview mode, and a lot of folks have been asking for more than a year, to have this fixed. Their answers have ranged from "It's a feature", to "It's on the list, with no ETA". Clearly, the company isn't listening to it's users. That's sad.

For Bitbucket, the issue seems to be revenue. They don't seem to feel the need to fix this serious documentation issue, and it can't be all that hard if GitHub has had it from the beginning. The people on the ticket have even quoted (and linked) the Markdown spec as well as how GitHub is doing this - and nothing has changed.

I can appreciate that this is their choice. Mine is to use it only if I have to, and realize that their idea of support is "No, thanks", and accept what is, over any hopes of what might be.

I had to update my work MacBook Pro, and I got the dreaded "login" Keychain problem because I guess the tools they created/used to update my local password were not up to snuff, and now that I've created a new KeyChain, I've also found that they have just horribly managed this laptop to the point that I'm surprised that it works at all.

It's this kind of lame administration that really makes me upset. They bought this bill of goods from some enterprisey shop about how to "Easily" and "Securely" administer their laptops, and rather than just teach people, or make it clear what to do, and not do, they have this crap with no way to remove it for those of us that simply don't need it.

It's frustrating. Yet in the end, I have to follow the advice I gave a friend just this morning... take it easy, this is just another indicator that this is not the place you really want to work. Simple.

I'm not a fan of OAuth2... not in the least. It's excessively complicated, it requires call-backs, and in general it's no more secure than anything else, it's just more complicated. Add to that there's no really good library for it as the Google folks keep changing things, and you have something that's always going to require hacks... always going to require fixing, and never going go provide a seamless way to authenticate on a remote system.

But that's just an opinion. I have had to make it work at The Shop, and when I finally got it to work, I wasn't about to let this evaporate into the ether... I needed to make a gist of it, and document what I was doing so that I could come back to this and be able to remember it all at a later date.

The State of OAuth2 Clojure Libraries

The first really depressing thing was that there seemed to be no decent OAuth2 libraries for Clojure. And while there seemed to be a lot of forks of the clj-oauth2 library - but many, like the original, were years old - and they didn't work. Not even close. Now I'm not silly enough to think that the spec changed, but I do believe that Google changed things on it's end to make it more secure, and in so doing, broke all the clj-oauth2 work, and it's derivatives.

Still, there is the code I can look at. And some have pulled in the features that are needed, and so it's not impossible to make this thing work... though it's likely to take a lot of time.

The project.clj File

When I was able to get something working, I made a gist of all the important files, so that I could include them here as reference. I also wanted to post the link to the #clojure room in IRC because one of the guys there gave me a hint as to which library to use. He wasn't right, but he was close, and that's all I needed.

The project.clj file has all the versions of the libraries I used:

What I found was that this version of clj-oauth2 had the most complete mapping of the data coming from Google, which included the expires-in time - which I think I still may be able to put to really good use soon. While it didn't have the functions to renew the access-token, it turns out that it's not hard to write, and I pulled that from another fork of the master project.

The server.clj File

OAuth2 still requires that the user go to Google on a redirect, and then the call-back from them is where we get the first bit of the authentication data. I'm not convinced that this is at all necessary, but it's how things are. Given that, we needed to have the server.clj have an endpoint /google that gets redirected to the right place at Google for the user to login and accept the app.

There is also the callback endpoint, and then a few that return the token data, and renew the token. Nothing special, really, but the targets for the OAuth2 are really important, and it's just sad that we have to have them in the first place.

The dcm.clj File

The final piece is really the meat of the problem.

We start off with the Missing Functions in the clj-oauth2 library, and then jump right into the static config for our application. These are all generated by Google, and you can get them from Google when you register your project/client.

We then have the authentication and re-authentication functions, which took an enormous amount of time to get right, but don't look overly complex in the least. Lovely.

Finally, we have a few calls to test that we got the user profile information properly, and that we can make subsequent calls to Google and get the data requested. It's not a lot, but it works, and it proves that things are working up to that point.

In the end, I'm glad I have it all done, and I'll be integrating the Custos Server in as a secret store of the credentials soon. Then I'll be using redis as the back-end and then pulling data from Google and loading it there. All this is a complete, stand-alone, back-end data collector for the ad messaging data for a client from Google.

Not bad...

Every now and then I remember all the work I did in college, and at Auburn University with SPICE and MAGIC. These are tools that I just can't ever forget, and all the time I spent on the terminals in the Terminal Rooms in the EE building... well... yes, let's just say they were some very Good Times. And of course, that was a good 30 years ago. Wow. In the subsequent years, MacSpice is around, and it's got all the models, and all the capabilities that the old SPICE had - plus some amazing plotting.

In fact, I'm sure my retina MacBook Pro is probably more powerful than all the computers the EE Dept. had in 1988, and it shows. I can start with this very simple LPF circuit:

  Four double-pole, low-pass, LC-filters
  VIN 1 0 AC 1
  * Q = 0.5
  R1 1 2 200
  L1 2 3 10mH
  C1 3 0 1uF
  * Q = 1
  R2 1 4 100
  L2 4 5 10mH
  C2 5 0 1uF
  * Q = 2
  R3 1 6 50
  L3 6 7 10mH
  C3 7 0 1uF
  * Q = 4
  R4 1 8 25
  L4 8 9 10mH
  C4 9 0 1uF
  .AC DEC 100 100hz 10khz
  .END

and then in the MacSpice console:

  MacSpice 1 -> source lpf.ckt
 
  Circuit: Four double-pole, low-pass, LC-filters
 
  MacSpice 2 -> run
  On line 2:
      vin 1 0 ac 1
      Warning: Source 'vin' has no DC value specified, 0V assumed.
  MacSpice 3 -> plot vm(3) vm(5) vm(7) vm(9)

and we see:

It really is amazing what these tools can do, and they continue to bring a smile to my face. What a treat.

I've been doing a little work on the Clojure CryptoQuip solver by adding a new RESTful endpoint for the server that takes the quip and the clue and solves the puzzle. It's not all that hard, but it's a POST call, as the body of the POST is JSON, and I needed a tool to hit my server to make sure that I had it all working. Enter Postman.

This is just an amazing tool for Chrome. It's a very singular task, and it does an excellent job of making sure that you can make all the calls you need with all the headers and arguments and get back what you need - and even keep a running history of all the calls to re-do should you need them.

Clearly the folks that wrote this knew what they needed and spent quite a bit of time on it. I have yet to find a hole, a bug, or a limitation. It's just good, solid working code. And it's free. Wow. Impressive.

I looked at several of the OS X tools on the App Store, and even on the web and the nice ones were more than $20, and most weren't nearly as nice as Postman. If you need something like this, you need it, and this is as good a tool for the job as you can get.

OK, this isn't something that typically happens, but today I got an email from the paralegal assigned to my lawyer for my divorce. Typically, I hate getting emails because it's either bad news, or a bill. Some would argue they are one in the same, but I see the bills as slightly less depressing than the "regular" bad news I get about the divorce.

So I got an empty email from her, so I wrote back:

This email is empty - I have no idea what this is about.

Please provide me a little context.

and she replied:

Ha! I was adding to my previous email that I could email you record (in several sections) AND deliver the paper copies so you don’t have to print it out and you will still have electronic copies. OK?

I had no idea whatsoever what she was talking about, but clearly she thought this was funny. She still didn't know that she was talking to the wrong person. So I went along with the joke, and wrote back:

I’m sure to you this all makes perfect sense, but to me I’m still thinking “What is this thing that Beth wants to get to me?” and “Why do I need paper copies and electronic ones?”

Maybe it’s an invitation to a party?

Maybe it’s a movie - Oh… I do love movies. Spy ones are good… I like Comedy too.

Maybe it’s a sandwich? No… well… maybe - a paper sandwich?

So I must not have communicated clearly… my fault… I do that a lot.

Let’s try again...

What is this email about?

What do you need to get to me? And why?

Is it important, or just informational?

I think if you answer these, I’ll have a lot better idea what you’re talking about. No guarantees, mind you, but I think I’ll have a better handle on things.

I could go on, and we did for another round of emails until she realized that she was mailing to me, and not another "Bob B" that she needed to be emailing. Then she picked up the phone and was quite apologetic. I thought it was at least a little amusing, but clearly she seemed to think the lawyer(s) would not find this comical - in the least.

I feel for her. An honest mistake is going to be a source of grief and embarrassment for her for the next day or so. I understand that lawyers are meant to be flawless, but we're all people, and we all make mistakes - big and small. I hope they don't give her too hard a time.

OK, I've spent several hours with Swift, and it's pretty close to Ruby, but it's not the same in that the String is still a class, and that means that it doesn't have the duality that a string in Ruby - or clojure - has. This makes a lot of the functions in the CryptoQuip solver a lot harder.

Little things, too - like finding the distinct characters in a String. In clojure it's easy:

  (count (distinct "rabbit"))
  => 5

and even ruby it's nearly as easy:

  "rabbit".chars.to_a.uniq.count
  => 5

but in Swift, it's nasty:

  func distinct(list: [Character]) -> [Character]  {
    var buff = Dictionary<Character, Character>()
    for x in list {
      buff[x] = x
    }
    return buff.keys.array
  }
 
  let src = Array("rabbit")
  distinct(src).count

I did find a StackOverflow idea that does it, but it's a function that basically builds up a new list by adding one element at a time, and then checking before each add that the new element doesn't already exist in the collection. Since both these collections are Arrays, this is a linear search over and over and over again, and it's a mess.

I'm sure Swift has a ton of nice features, and it's quoted as faster than ObjC - which is nice, but issues like this make me realize it's a 1.x language and that the builders haven't had the time to go into the same use-cases that they have for other code bases, and fill in all these kinds of features that will make it a lot easier to write more complex systems.

For now, I'm content to hold off on more Swift work. It's just not where I'd like it to be.

UPDATE: I added in the function that I wrote, and it works, but Holy Cow! it's a mess. You can't easily make it one call:

  distinct(Array("rabbit")).count

because you have to worry about the mutability of the Array, and that causes warnings in the code... it's just not ready for all the things you need to do. Soon, hopefully.