What's it all about, Alfie?

Today I've spent a good bit of time working on a new project - a lisp-like language in C++ for processing scripts in as efficient manner as possible. The idea was really from some work I've done in the past as well as talking to some nice guys at a start-up that are looking to do the same kind of thing. What I was thinking was that I'd put something together, and point them to it so they could get an idea of how I'd do it. Then, they are free to use it or not, and I've done my part to help them out.

I interviewed with them, but I don't think I'm going to be getting an offer, or taking the offer, as it's likely to be lower than what I have from another place, and it's a small shop, and therefore, a significantly greater risk. But I want to help them, and this is what they were hoping I'd be doing if I joined the group.

The first steps were pulling up the ideas I had from the original code, but there were a lot of things about that code that I didn't like - specifically in the area of performance. I'd like this version to be as fast as possible as it's going to be working on very large data sets - typically time series data, and so the type of data is somewhat restricted, but the amount is enormous.

I got the first two classes built: value and then variable (a value with a name). Now I need to step back a bit and work on the rest of the component design: expressions, functions, etc. The question will be: Should I break out the evaluation methods from value into a different base class? I'm not sure if I need to do this, or if it's really going to benefit the design in any way.

Certainly, it'd be nice to have all the "evaluate-able" objects based off one super class, but I'm not sure that I need that level of flexibility. After all, a function takes a series of values as arguments, and produces a value. An expression is really a function and/or values in some relation to one another. It's not like I need to have this all that different from what it is now.

For instance, I can base the function on the value and have the result value put into the ivar for the value as a cached value to speed up the resulting calculations. That would be nice. Then expressions are something else… maybe they are just relations of values. Not sure.

In any case, I've got a start, and it's going to take a lot more work to get something ready to put on GitHub and show to the guys at the start-up. But I'll get there.

I have been wishing that GitHub for Mac would properly show my Gravatar since the first release so long ago. The problem is that on GitHub, my Gravatar shows up just fine:

but on GitHub for Mac, it's all the default I have no idea who you are… image:

The problem stems from the issue that Gravatar assumes that the email is all lower-case before it's sent to Gravatar for image lookup. This is compounded by the fact that Gravatar does not allow you to have two emails registered for the same Gravatar that only differ by the case in the email. This means that drbob@themanfromspud.com and drbob@TheManFromSPUD.com are totally different Gravatars, and the latter will never be found.

This is highlighted by the PHP code snippet that I found for successfully getting a Gravatar:

  $out = "http://www.gravatar.com/avatar.php?gravatar_id=".md5(strlower($email_addr));

I'm willing to bet that GitHub for Mac is not doing the lower-casing of the email address before it's taking the MD5 hash and sending that to Gravatar. To test this, I changed the user.email parameter on my MacBook Pro and did a few checkins:

The commits I did today have the lower-case version of the user.email setting in the commits, but the older ones don't. This is a clear indication that I'm right. Now it's possible, I suppose, to go back and completely update all the repos I'm using, but I think that's a bit excessive. I'd like Gravatar to allow for mixed-case, but they just take the gravatar_id, and it's already MD5 based at that point.

Nope, this is a GitHub for Mac issue, and they need to lower-case the email address before they send it to Gravatar and then all will be well. I've sent in the request to GitHub, and we'll see what they have to say. I'm hoping they fix this, I do hate to see the little grey shadow for all my checkins.

UPDATE: Great News! I heard back from the GitHub for Mac guys:

  From: Josh Abernathy (GitHub Staff) 
  Subject: GitHub for Mac - Gravatar Mixup

  Hi Bob,

  Ahh, thanks for tracking that down for me. I've created an issue for it.

  Thanks!

so it looks like it's going to get fixed in an upcoming release of GitHub for Mac. That's such wonderful news as it means that all my avatars will be me and not the gray shadow. Super nice!

This morning I decided to check out the Pull Requests on GitHub. Specifically, I've always wondered how they work, and what the interface is, and I've had a nice little example waiting for me in the XMLRPC library that I'm using for SyncKit. So now is as good a time as any.

The core of adding ARC (Automatic Reference Counting) support for a Cocoa project is that ARC support is really compiled into the code. ARC is basically the LLVM correctly identifying when retain, release and autorelease need to be called, and then putting that code into the compiled executable without it having to be in the source.

It's slick, clean, and makes the code a lot less prone to memory issues. But there are things you can't do in ARC code that you can do in retain/release code - like compare-and-swap atomic operations. This is bad for the really low-latency stuff, but when you get into iOS, the ARC stuff is really a much easier way to make sure your app - and not you, has good, clean memory management.

I picked up the following simple fix for ARC support from Gus of FlyingMeat fame, and his FMDB project on GitHub:

  #if ! __has_feature(objc_arc)
    return [NSMakeCollectable(temporaryUUIDString) autorelease];
  #else
    return (__bridge_transfer NSString*)temporaryUUIDString;
  #endif

Because the LLVM compiler knows what the illegal methods are, it's a simple matter of going through all the code and correcting it a line at a time. It took me about two hours to do, and then I had something that worked great for AppKit and UIKit. Sweet.

So this morning, I forked the repo on GitHub, then cloned it to my MacBook Pro, made the changes and cleaned up the whitespace a bit, and then checked it all in and pushed it up to GitHub. Now was the time for the Pull request.

Happily, it's very simple. GitHub knows the commits you have done since the fork, and it's simply a matter of sending those commits to the original project maintainer, and it's up to him to incorporate them or not. You can write a nice set of comments to go along with the pull request, and I did, but I'm not sure that's all that necessary for something this simple. But is was fun.

It's off, and now it's up to Eric to see if he wants to support this. I've got my forked repo now, so it's no big deal to me, but it'd be nice to be able to stay up to date and see the code get better because of my involvement.

Sweet!

UPDATE: It was accepted!

This is why I love public domain coding. I can work to make something better, and the author looks at it and says Sure, why not? This is a great thing we are doing - those of us that can code well… and we need to do more of it. Give more than you take. That's a great motto.

OK, it's hard to imagine that just a month and a half ago, $2 billion lost due to improper hedging was thought to be a horrible thing… now it's looking like it might go as high as $9 billion. Amazing. I thought I'd heard the last of this horrible sham, but it appears not.

JP Morgan said they had $2 billion to waste, and still make a profit, now they have to pony up an additional $6 billion or more to get out of the horrible positions they got themselves into thinking they were hedging their legit positions. How much money can JP Morgan afford to loose and not be seen as the biggest pirate of the seven seas?

I've been in finance for 16 years now, and I've learned a lot, and been surprised over and over again. This is just the latest in a series of amazing things that have come up in this industry in my time. If I land a job outside this industry, I won't be sorry to leave it. I've met a lot of nice people, and yet this is an industry that's begging to be regulated even more heavily than it already is.

This morning I was putting together a GitHub pull request for the XMLRPC library I'm using in SyncKit, and I realized that I'm really tired of the tab size on the git diff being 8 when all my code uses a tab size of 4. So I finally started googling to find an answer, and it turns out that the presentation of the git diff is really nothing more than the Unix command less. Interesting. Makes sense.

So you can make the tab size 4 by simply running:

  $ git config --global core.pager 'less -x4'

and if you omit the --global argument, it'll set it just for the repo you are in. This will be great when I have a repo with a tab size of 2, for instance. I can set the global to 4, and then override it as needed in the repo. Very nice.

Once again, Git rules.

I have taken to using Safari for my GitHub views, and the problem there is that I'm typically looking at landscape pages in Safari, and not wide screen pages. But for GitHub, it's much nicer to look at things wide screen so that we get the entire width. What I'm left with is two different preferred sizes, and Safari will remember the last size you had, so it make sense to look into a way to make things a little easier for setting these sizes. Enter AppleScript.

It's actually pretty simple, once you know the things you can ask for, and get. The script to put the front-most Safari window into landscape mode is:

  tell application "Safari"
    activate
    set myPos to bounds of front window
    set x to item 1 of myPos
    set y to item 2 of myPos
    set bounds of front window to {x, y, x + 601, y + 629}
  end tell

placed into your ~/Library/Scripts/Applications/Safari/ directory.

To set the front-most Safari window to wide screen mode, I simply used a different geometry:

  tell application "Safari"
    activate
    set myPos to bounds of front window
    set x to item 1 of myPos
    set y to item 2 of myPos
    set bounds of front window to {x, y, x + 678, y + 468}
  end tell

With these in the aforementioned directory, I can use them to resize the front Safari windows very nicely. Sweet.

I'm not sure if I'm going to be doing a lot of AppleScript, but it's nice to have when you need to throw simple things like this together.

While I like Ubuntu 12.04 a lot, the lock screen on the screen saver is a little too much authentication for me. It's a laptop, and it's on my desk, so it's not going anywhere. But I couldn't find an easy way to get rid of it. Until this morning. Yeah!

Turns out it's a simple preference command:

  $ gsettings set org.gnome.desktop.screensaver lock-enabled false

And with this run once, the screen save is still active, but when I hit a key to wake it up, I don't have to type in my password to see the desktop. Nice! This is a really nice desktop linux system - in addition to being a great server platform. Very impressed.

This morning I noticed that yesterday while I was at the interviews, the Google Chrome team release 21.0.1180.11 to the dev channel. The changes are sounding pretty routine these days: new V8 javascript engine 3.11.10.12, more Retina (HiDPI) enhancements for the new MacBook Pros, and several other crash fixes. Not bad for an update. I'm pleased that they are keeping the speed up after those few sluggish releases, so we'll see what they have planned for the 22.x series.

It's been a long day, and I'm not feeling exceptionally great about the possible outcome of the day's work. It all had to be done, but I'm worried about what might, and what might not, come of all the talking and walking.

Today I had two interviews at places in Chicago, and so I got up this morning and went forth, determined to land not now, but two, offers, and be able to pick my favorite of the two. I had a more 'corporate' interview first, and I believe they are going to offer me a position, but I could be wrong. It's certainly happened before. And even if they offer me the position, I'm concerned that it'll be so low I won't be able to take it without also having to some up with some other source of income to supplement it - or move and drastically reduce our living expenses.

The second was with a start-up, and while I really liked the guys, and can see myself working with them on this venture, again, I don't see them coming up with the money I told them I'd have to have to work there. I can appreciate why they don't have the money, but then I'd really almost have wished that they hadn't have me talk to them if they knew they couldn't pay me. It just gets kinda dissappointing.

In the end, I don't know anything at this point, and I just need to sit tight, follow other leads, and see where things lead. Who knows? Maybe they will surprise me both.

That would be really nice.

This afternoon I decided that maybe it was time to see if I could get ZeroMQ built and running on my MacBook Pro running OS X 10.7 as well as my Ubuntu 12.04 laptop. I'm thinking it might be nice to write a few little test apps again with the latest ZeroMQ APIs between the machines to make sure that I have everything I need - should it come to that and I need to implement a little ZeroMQ action into DKit, or some other library.

The first step is downloading it from the ZeroMQ site. I picked the POSIX tarball as it's the one with the created ./configure script, and I needed that in order to get things kicked off.

Next, we try to build it on OS X 10.7 and Ubuntu 12.04. There are a few changes that have to be made to the OpenPGM code in order for it to compile on OS X. They are basically the includes needed, and not allowing duplicate definition of values.

In ./foreign/openpgm/build-staging/openpgm/pgm/include/pgm/in.h :

Replace:

/* sections 5 and 8.2 of RFC 3768: Multicast group request */
struct group_req
{
  uint32_t    gr_interface;           /* interface index */
  struct sockaddr_storage gr_group;   /* group address */
};
 
struct group_source_req
{
  uint32_t    gsr_interface;          /* interface index */
  struct sockaddr_storage gsr_group;  /* group address */
  struct sockaddr_storage gsr_source; /* group source */
};

with:

#ifndef __APPLE__
  /* sections 5 and 8.2 of RFC 3768: Multicast group request */
  struct group_req
  {
    uint32_t    gr_interface;           /* interface index */
    struct sockaddr_storage gr_group;   /* group address */
  };
 
  struct group_source_req
  {
    uint32_t    gsr_interface;          /* interface index */
    struct sockaddr_storage gsr_group;  /* group address */
    struct sockaddr_storage gsr_source; /* group source */
  };
#endif  // __APPLE__

In ./foreign/openpgm/build-staging/openpgm/pgm/sockaddr.c :

Replace:

#include <errno.h>
#ifndef _WIN32
#    include <sys/socket.h>
#    include <netdb.h>
#endif

with:

#include <errno.h>
/* Mac OS X 10.7 differences */
#ifdef __APPLE__
#    define __APPLE_USE_RFC_3542
#    include <netinet/in.h>
#endif
#ifndef _WIN32
#    include <sys/socket.h>
#    include <netdb.h>
#endif

In ./foreign/openpgm/build-staging/openpgm/pgm/recv.c :

Replace:

#include <errno.h>
#ifndef _WIN32

with:

#include <errno.h>
/* Mac OS X 10.7 differences */
#ifdef __APPLE__
#    define __APPLE_USE_RFC_3542
#    include <netinet/in.h>
#endif
#ifndef _WIN32

The final change is to the ZeroMQ source itself:

In ./src/pgm_socket.cpp :

Remove lines 88-92, make this:

pgm_error_t *pgm_error = NULL;
struct pgm_addrinfo_t hints, *res = NULL;
sa_family_t sa_family;
 
memset (&hints, 0, sizeof (hints));
hints.ai_family = AF_UNSPEC;
if (!pgm_getaddrinfo (network, NULL, &res, &pgm_error)) {

look like this:

pgm_error_t *pgm_error = NULL;
if (!pgm_getaddrinfo (network, NULL, addr, &pgm_error)) {

At this point, we can get ZeroMQ to compile on Mac OS X 10.7 as well as Ubuntu 12.04. But there's a slight wrinkle… while I'm fine with the linux library being a 64-bit only architecture:

  drbob@mao:~/Developer/zeromq-3.2.0$ file src/.libs/libzmq.so.3.0.0
  src/.libs/libzmq.so.3.0.0: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/
  Linux), dynamically linked, BuildID[sha1]=0x71a160f17833128c864811b25942cdacdb54
  f6d0, not stripped

I'd really like the Mac OS X 10.7 dynamic library to be Universal, with both 32-bit and 64-bit architectures in it. Currently, it's 64-bit only:

  peabody{drbob}82: file src/.libs/libzmq.3.dylib
  src/.libs/libzmq.3.dylib: Mach-O 64-bit dynamically linked shared library x86_64

Hmmm… it's really amazing why some folks choose to write Makefiles in a way that doesn't allow you to use multiple -arch arguments. There really aren't all that many way to mess this up, but it seems that the OpenPGM and ZeroMQ guys have done it really pretty nicely. I can't simply remove the offending compiler flags and add in the necessary -arch i386 -arch x86_64. So I have to make it twice: once for i386 and again for x86_64 and then use lipo to stitch them together.

In the Makefiles, I added the -arch i386 command to the CFLAGS and CPPFLAGS variables after successfully building the 64-bit version of the libraries. I then did a simple:

  $ make clean
  $ make

and then when I looked at the resulting libraries they were 32-bit. I then just created the Universal binary with the lipo commands:

  $ lipo -create libzmq.3.dylib .libs/libzmq.3.dylib -output .libs/libzmq.3.dylib
  $ lipo -create libpgm-5.1.0.dylib .libs/libpgm-5.1.0.dylib -output \
    .libs/libpgm-5.1.0.dylib

Now I can copy these away and install them in the right location for usage. Typically, I'll do a make install and place everything into /usr/local/ and then copy these Universal binaries over the installed mono-binaries and be ready to roll.

UPDATE: I can get ZeroMQ to work with the simple transports, but the OpenPGM encapsulated transport fails miserably. I'm looking at the code and I'm becoming convinced that no one is really using or testing this code. It's a mess and there's no way some of these method calls are going to work. It's all in the PGM portion, so if they stay away from that, it's fine. But that's not good enough for me. So I'm giving up for now. I have asked a few things in IRC, but there aren't any responses (yet), and I don't expect to get any about the OpenPGM part.

I think it's dead, and I need to just move on. Sad to see, but it happens.

[6/21] UPDATE: after a lot of work trying to get the OpenPGM 5.1.118 code to work, I see that the developer knows about the problems, and is just not ready to support Mac OS X 10.7 at this time. I can respect that, but the things he had to have done to make his code this non-standard must be pretty wild. So there's nothing I can do at this point. OpenPGM is a deal-breaker, and that's the entire reason I wanted ZeroMQ.