What's it all about, Alfie?

One of the things that I've really missed in the latest version of Mac OS X is the utility to set the default version of the JDK to use. Well, with all the clojure coding I've been doing, I have typically stuck to JDK 1.7, but recently, the auto-updater from Oracle has them shipping JDK 1.8. Yet this doesn't effect the command-line utilities. This is a pain in the neck.

So first, I got JDK 1.7.0_75 on all my machines - it's going to be the last version of JDK 1.7 that I'll get as it's off the auto-updater. Then I was sure that everything was OK, I also got the complete JDK 1.8.0_40 - which is what the auto-updater downloaded for the JRE, but not the JDK. Yes, annoying.

At that point, I started googling about what to do. Turns out, JAVA_HOME is all I need to worry about. The java, jar, java apps in /usr/bin are all respectful of JAVA_HOME. Then I found this function that I added to my ~/.bashrc:

#
# Clever trick to leverage the /usr/bin/java commands to take advantage
# of the JAVA_HOME environment variable and the /usr/libexec/java_home
# executable to change the JDK on-the-fly. This is so easy I'm amazed.
#
function removeFromPath() {
    export PATH=$(echo $PATH | sed -E -e "s;:$1;;" -e "s;$1:?;;")
}
 
function setjdk() {
    if [ $# -ne 0 ]; then
        removeFromPath '/System/Library/Frameworks/JavaVM.framework/Home/bin'
        if [ -n "${JAVA_HOME+x}" ]; then
            removeFromPath $JAVA_HOME
        fi
        export JAVA_HOME=`/usr/libexec/java_home -v $@`
#        export PATH=$JAVA_HOME/bin:$PATH
    fi
}
setjdk 1.7

At this point, I saw that the key command was /usr/libexec/java_home, and it had all the answeres I needed. I didn't need to update my PATH - just JAVA_HOME. I also could verify that I had all the versions of Java I needed:

  $ /usr/libexec/java_home -V
Matching Java Virtual Machines (7):
  1.8.0_40, x86_64:  "Java SE 8"  /Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/
                                   Contents/Home
  1.7.0_75, x86_64:  "Java SE 7"  /Library/Java/JavaVirtualMachines/jdk1.7.0_75.jdk/
                                   Contents/Home
  1.7.0_51, x86_64:  "Java SE 7"  /Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/
                                   Contents/Home
  1.7.0_45, x86_64:  "Java SE 7"  /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/
                                   Contents/Home
  1.7.0_13, x86_64:  "Java SE 7"  /Library/Java/JavaVirtualMachines/jdk1.7.0_13.jdk/
                                   Contents/Home
  1.6.0_65-b14-466.1, x86_64: "Java SE 6"  /System/Library/Java/JavaVirtualMachines/
                                            1.6.0.jdk/Contents/Home
  1.6.0_65-b14-466.1, i386:   "Java SE 6"  /System/Library/Java/JavaVirtualMachines/
                                            1.6.0.jdk/Contents/Home

And then with a simple command - in the shell of my choice - I can set - and reset the JDK I use. It's very click:

  $ setjdk 1.7
  $ javac -version
  javac 1.7.0_75
  $ setjdk 1.8
  $ javac -version
  javac 1.8.0_40

Everything else works like a charm. This is a handy little tool to have, and I'm sure I'm going to use it often as I migrate from JDK 1.7 to 1.8.

Today I was talking with Carl, the friend I was doing a little side-project with in Clojure, and it was pretty clear that he didn't understand what the next steps were for the project, and while I tried to explain it to him, he's thinking in terms of cross-sectional probability functions, and I'm thinking this is all a massive Monte Carlo simulation system where the PDFs for the individual events are what you tweak, and then you run the simulation for a year of time, and see what happens. It's very simple, and while it doesn't give you direct control over the things out might want to control, it does give you the ability to see how individual actions - the promotions from one step in the sales pipeline to the next, impacts revenue.

Anyway, so I wanted to make sure that it was all running nicely, and I was using Gorilla REPL for the REPL as it has nice graphing as well as being a generally good REPL, but as I was making changes to the code and refreshing the page, I wasn't seeing what I expected to see.

Then it hit me - Gorilla REPL wasn't reloading the code.

Ouch.

I then switched to using the standard REPL with Leiningen, and everything worked fine - then firing up Gorilla REPL and reloading the page showed that it was now working just fine. Lesson learned - Gorilla doesn't reload the code when you re-execute all the code.

This morning I thought I'd just check on the status of Apache + PHP + Postgres on OS X 10.10 as it was likely that in the update from 10.9, Apple had changed things, and sure enough, they have. But it wasn't all them - Apache has some changes in it as well, so to be fair, getting this all to run is a little more complex than you might imagine, but it's still not all that hard.

Getting PostgreSQL 9.3

As with the previous post I'm using Homebrew and that's made the entire operation a lot smoother. There's no 32-bit/64-bit issue as Homebrew does both, and it builds in on your box, so again, a lovely customized solution with a simple command:

  $ brew install postgresql

It even has a nice, simple informational blurb about how to start/stop and upgrade. Very nice. But now that it's on the box, and ready to roll, let's add in the PostgreSQL support to the PHP 5.5.14 that's installed with Mac OS X 10.10.

Activating UserDir in Apache 2.4.9

The first thing I noticed is that the UserDir extension is not enabled by default, so we need to get that going right away. This enables the code to be run from the development directories, and that's a big time-saver. First, we need to enable the UserDir module in Apache, and then make a specific config file for the user in question. Start by editing /etc/apache2/httpd.conf and line 166 needs to be uncommented to read:

  LoadModule userdir_module libexec/apache2/mod_userdir.so

and then similarly on line 493 uncomment the line to read:

  Include /private/etc/apache2/extra/httpd-userdir.conf

At this point, you need to make sure you have at least one file in the /etc/apache2/users/ directory for each user, like: drbob.conf:

  <Directory "/Users/drbob/Sites/">
      Options FollowSymLinks Indexes MultiViews
      AllowOverride None
      Order allow,deny
      Allow from all
      Require all granted
  </Directory>

where the last line - Require all granted is new as of Apache 2.4, and without it you will get errors like:

  [Thu Dec 18 10:41:32.385093 2014] [authz_core:error] [pid 55994]
    [client fe80::7a31:c1ff:fed2:ca2c:58108] AH01630: client denied by server
    configuration: /Users/drbob/Sites/info.php

Activating PHP in Apache

The mext thing to do is to activate PHP in the supplied Apache 2 with OS X 10.10. This is line 169 in the file - /etc/apache2/httpd.conf and you need to uncomment it to read:

  LoadModule php5_module libexec/apache2/libphp5.so

and then add a new file called /etc/apache2/other/php5.conf and have it contain:

  <IfModule php5_module>
    AddType application/x-httpd-php .php
    AddType application/x-httpd-php-source .phps
 
    <IfModule dir_module>
      DirectoryIndex index.html index.php
    </IfModule>
  </IfModule>

which does all the other PHP configuration in a separate file to make upgrades easy.

Building in PostgreSQL 9.3 extra for PHP 5.5.14

At this point we need to get the source code for the exact version of PHP that Apple ships. This is pretty easy by getting the PHP 5.5.14 source from a mirror and unpacking it into a directory. We then just run the following commands:

  $ cd php-5.5.14/ext/pgsql/
  $ phpize
  $ ./configure
  $ make

at the end of this process, you'll have a file: php-5.5.14/ext/pgsql/modules/pgsql.so and that needs to be placed in the right directory and referenced in the php.ini file.

For Mac OS X 10.10.1, this is accomplished with:

  $ sudo cp modules/pgsql.so /usr/lib/php/extensions/no-debug-non-zts-20121212/

and then edit the /etc/php.ini file to include this new extension by adding the line:

  extension=pgsql.so

Thankfully, the extension_dir is already set in OS X 10.10.1, and we no longer have to set that.

Finishing Up

At this point, a simple restart of apache:

  $ sudo apachectl restart

and everything should be in order. Hit a URL that's a static file with the contents:

  <?php
    phpinfo();
  ?>

and you should see all the details about the PHP install - including the PostgreSQL section with the version of Postgres indicated:

All is working once again!

I read with a giggle this article off Slashdot about the problems companies are having making Hadoop actually work in their environment. This is not in the least surprising to me. Hadoop is a nice, distributed, storage environment, but so is a very nice SAN. Both can store a lot of files, and do it very quickly, with redundancy, but one is putting entire computers on each "disk", while the other is allowing the disks to just be... well... disks.

The promise (hype) of Hadoop is that by distributing the computing power like the storage, the map/reduce jobs can be done fast and easily and you get old-style SQL performance with as much online storage as you can muster.

But the truth of the matter is far different than this, as this article in the WSJ attests.

Hadoop is nice, but it's not fast. It's good for lots of storage space, but so is a SAN. It's nice to do small map/reduce jobs, but so is CouchDB. But you can't scale CouchDB to any size you want, either. There will always be limits.

I know folks that are looking at Cassandra, and really like it. It's not as general-purpose as Hadoop, but it's targeted at the problem of massive storage with SQL access. Advocates of Hadoop will say Use Spark - get SQL that way! and that's possible, but then that's not Hadoop, is it?

You can use many different caching schemes to make any storage scheme work faster, but the promise of Hadoop was that you wouldn't need them. That's where it fell over. I have no doubt that Hadoop can do a lot of really good things in a lot of very specialized environments, but it's not the silver bullet worth $2 billion. It's a nice open source map/reduce like CouchDB, but on as many boxes as you want. It's nice... but it's not what people hoped it would be.

Too many distractions in systems development these days. Just too many.

Over break I'm going to be working at home on a few little fun projects, and with my two Thunderbolt displays, I was thinking I might need a hub of some kind, so I went to look for one on the Apple Store web site - just to see what was out there, and what they might cost - if I shopped in town.

I honestly didn't find anything, and then I wondered What's on the back of the monitor? to see if I could chain these together? Turns out, you can. The guy working across from me as two, and they are chained, and I've seen them work just fine. There's even a YouTube video on the set-up.

Very nice!

This will be just what I have when I get started on Saturday morning - if I wait that long. Apple stuff is pricey, yes, but you do really get what you pay for, and this is no exception: one cable to the laptop, driving two 27" displays. Sweet.

This morning I was looking into yet another Adium problem with MSN Live, and I opened up the Console.app and noticed a lot of messages like this:

  12/18/14 4:08:22.886 AM com.apple.xpc.launchd[1]:
    (homebrew.mxcl.postgresql[52026]) Service exited with abnormal code: 1

and when I tried to check if Postgres was running:

  $ psql -l
  psql: could not connect to server: No such file or directory
          Is the server running locally and accepting
          connections on Unix domain socket "/tmp/.s.PGSQL.5432"?

so it was clear that something was wrong with postgres on my laptop.

To be fair, I haven't really messed with this since going to Yosemite, so I hit DuckDuckGo and it reported this post on StackOverflow, which says that in the Yosemite upgrade, several empty directories were deleted, and all I needed to do was to re-create them and restart postgres.

So I:

  $ launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
  $ cd /usr/local/var/postgres
  $ mkdir pg_tblspc
  $ mkdir pg_twophase
  $ mkdir pg_stat_tmp
  $ chmod 700 pg_tblspc pg_twophase pg_stat_tmp
  $ launchctl load ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist

and then I could see all the databases on the laptop, and the Console.app showed that things were running just fine.

While I understand that postgres is inefficient as it's running all the time, I still love this database, and it's worth the cycles to have it on my development laptop because it scales to the server room so bloody well.

Problem solved. Whew!

While it's no GitHub, I've been working with Bitbucket for a few projects with a friend, and I have to say it's not bad at all. I happen to like the helpful hints GitHub offers on making a new repo, but the help Bitbucket has is very fair, and reasonable. It's got a little different UI, which is fine, and it works pretty much the same so that you don't have to come up to speed with a new workflow if you're switching back and forth from Bitbucket to GitHub.

I looked at Bitbucket a long time ago, and it wasn't nearly as similar to GitHub as it is now, and that makes a lot of sense. It's got Pull Requests, and it's got a very nice source view:

Many will say It's just like GitHub's - which is a good thing. Face it - you copy what's a great design and works well. This is one of those cases.

The thing I like about Bitbucket is the free repos for small teams - up to 5 people. In short, they have a different approach than GitHub - repos are free, people are charged. It's a nice approach because it means that non-Open Source projects that you work on with a friend can be private without a cost. GitHub charges for the first private repo, and I don't begrudge them that - I'm a paying GitHub user. But I like the different approach because it gives small groups a leg up.

I'm not going to stop using GitHub, but it's nice to see that Bitbucket exists and seems every bit as good as GitHub for the task. It's something I'll remember.

As I didn't have a lot to do this afternoon, I decided to check on the state of X11 on OS X 10.10.1 - and it's actually not bad at all. I was able to get Quartz 2.7.7 from the download site, and it installed easily enough. All the normal things run, and it's a complete X11 we just need the apps - or in the case of SSH access, getting to a linux machine for the X11 apps on the remote host.

Because I got rid of my home machines with X11 a while back, I'm not sure how much this is going to get used any more, but it's nice to know that it works before I really need it, so that I know I can go to it in a time of need.

I've been using Dash for a while now. It started out with a really odd looking cat as the icon, and it was a little iffy in the beginning, but it got better - a lot better. Today, it's the best documentation viewer I have ever used. It's just stunning. And it's fast. I can't believe how fast it is, but maybe it caches everything in a docket in memory - who knows? But Wow... it's easy to use, expansive on all the docs it covers, and the speed and ease of use are simply hard to beat.

I have a set for Clojure coding which includes clojure, postgres, and Java, and I have one set for ruby - with ruby, postgres and Java (for JRuby). All told, it's amazing how access to the right docs immediately is so powerful.

I just updated my post about the conversion of the CryptoQuip Solver from Obj-C to ruby, and then for a piece of that to clojure, after having built that code in LightTable, and it just brought a smile to my face. The code was fun because closure allows me to do things that most languages simply don't, and it's always a simplification when that happens. In ruby I had to say:

  (pairs.count == ctc) && (ctc == ptc)

but in clojure, it's simpler:

  (= (count p) ctc ptc)

the = operator allows for any number of arguments - as does the other inequalities. Say I needed to have four numbers that all were ascending. That would be a hassle in ruby - maybe a sort, followed by the check that no two were the same, but in clojure it's as simple as:

  (< a b c d)

where it's only going to return true is d is greater than c is greater than b is greater than a. It's simple, and it's elegant.

But I also really like writing to friends about this stuff, and writing here as well. I've been recovering for almost two years now, and it's about time I work a little harder at just forgiving and moving on.

Because there's so much fun stuff to do, and if this little code block is all that exciting and fun to me, I've got a lot of fun ahead of me in a place that will feed my desire to solve problems.

And if I'm lucky enough to once again work for myself, then this blog is going to once again be open, and I'll be putting a lot more about solving problems up here as I think that's just amazingly fun stuff to share.