What's it all about, Alfie?

I have done a lot of work with redis, and of late, the most work I've been doing with it was to shard (aka cluster) many redis instances on a single box - and on multiple boxes. This isn't bad, but let's face it, it'd be great if redis could do this all on it's own - like it currently does replication.

Then I read:

Basically it is a roughly 4 years old project. This is about two thirds the whole history of the Redis project. Yet, it is only today, that I’m releasing a Release Candidate, the first one, of Redis 3.0.0, which is the first version with Cluster support.

Very nice! I'll be very interested in seeing how it works, and how it will scale with network load and if it's able to be configured for on-box vs. off-box connectivity. That could make a huge difference in the communication bandwidth.

Nice to see that it's almost here.

I spent some time this morning going over my Git commit logs trying to find out what I've been doing in the last two weeks - and then converting those into decent posts - at least as decent as I could make them in hind-sight. The reason for all this is the re-org, and the resulting group that's I've been thrust into. It's not a pretty sight - and I've seen this far too many times in my professional life to think it's going to end well.

I had a conversation with the larger group about why I see so many git force push messages in HipChat. I'm sure there's a reason, and it's part of some work-flow - I just haven't used that, and it seems pretty harsh to be used so often. Fact is, their reason was that it just made the commits "prettier". That they wanted to edit their commits like they edit their code.

I was silent because that was almost certainly one of the most insane things I've heard in a long time. Yes, by all means, let's risk the corruption of the git repo on the server because you didn't take time to think about what you were doing before you decided to commit it. Yeah, that's a good plan.

Better yet - let's make it part of the standard work-flow that you teach new developers. Yeah, that's a great idea. I just can't wait to start messing things up because I refuse to put forth the effort to think about what I'm doing prior to doing it.

In another meeting, one of the other developers felt that a Core Value of the new group should be that... and I quote: We all should have fun. Yeah... I don't even have to respond to this because unlink this developer, I understand the point of a public company, and that they really aren't in existence to make their employees happy - or make sure they have fun - they are in the business of making money.

I tried to say that fun is a great thing to have, but holding it up as the reason we're doing - and not doing - things is kinda crazy. That we don't have to have fun to get work done. It would be nice - but it's not necessary. Flew right over his head.

And the odd thing is that this is not a bad guy - well... I've heard he's a good guy, so I want to believe that my assessment of him to date is way off, but when he goes on about how I couldn't be more wrong... well... it's hard to give him the benefit of the doubt - or write it off to a miscommunication or something.

Nope, I'm in a group that doesn't seem to share a single value I have. They think it's acceptable risk to have git force push in their workflows, and they think that they have to have fun or they don't have to work. Wacky.

It's been hard to get things done, and if I took their lead, I wouldn't have done a thing - but thankfully, I didn't, and did a good chunk of work each day. In the end, I need off this group. I like the work, I just can't stand the management and co-workers any longer.

Today I spent a good bit of my day working on speeding up a few postgres stored procedures (functions) to get the speed up to the point that it's possible to do what we need to do in the time we have allotted. The original implementation was to create a temp table, populate it with the complete history of the time-series, and then filter it out based on the clever use of row_number() and picking just one value in a repeating sequence.

This was fine as long as the time-series didn't get too big. Then the temp tables were crushing us, and the time was insanely long. What really helps is to not make the temp table - but just get everything ready for a single SELECT where sub-queries do the caching for you.

Like I said, it was most of the day, but the difference was more like 50x to 100x difference - well worth the time.

This morning I was getting ready for a roll-out of some new code, and I didn't want to have to re-populate the production cache with a lot of data that I already had in my UAT cache - which has been running for more than a week, and populating data as it goes. The script isn't all that hard - is just runs through all the servers that need to be cloned, stops them, copies over the data file and then restarts the redis server.

  #!/bin/bash
 
  for b in {1..8}; do
    echo "working on redis_${b}..."
 
    echo "  stopping redis${b} server..."
    pid=`top -Unobody -c -b | grep redis_${b}.conf | awk '{ print $1 }'`
    sudo kill -9 $pid
 
    echo "  copying data from caliban over..."
    scp opt-analytics-redis4-uat.snc1:/var/groupon/redis_${b}/dump.rdb .
    sudo cp dump.rdb /var/groupon/redis_${b}/
    sudo chown nobody:nobody /var/groupon/redis_${b}/dump.rdb
    rm dump.rdb
 
    echo "  restarting redis_${b} server..."
    sudo /usr/local/etc/init.d/redis_${b} start
  done

Yeah, it's nothing special, but I'm trying to be a little more diligent on the posts, and this was kinda fun to write, and it works perfectly.

I've been looking at Atom as a potential replacement to Sublime Text 3 - my current editor, and it's getting closer, but it's not there yet. They have come up with several new themes - and at least one of them is very nice. They have put in the vertical space option, which was missing for me. They have even put in enough hidden defaults to make it possible to hide the tabs and the left-side bar.

So it's close. Really close.

But it doesn't remember it's state on exit. That's huge for Sublime Text 3, and something I just feel is important enough not to leave behind. I'm expecting the "find anything" that Sublime Text has, and BBEdit doesn't is coming - if it's not there now, but that really doesn't matter to me at this point. Lack of restart state is a killer.

But it's nice to see the GitHub crew aren't stopping with their initial release. And I've read that they are looking to add this back in - which will be nice. But they'll also have to do a lot about the memory footprint - Sublime Text 3 is a lot smaller footprint.

Today I had my first real run-in with a Process Over Progress at The Shop, and I have to say, this is really the reason that I enjoy leaving jobs. Nothing is a clearer sign of the decay of an organization than to have process seen as an end unto itself, and that things like the form of a document are more important than the content. Where the reviewers of a document can't be bothered to read the document they are reviewing, but still feel perfectly justified sitting in judgement of the author of the document.

I know this is a people-problem, which is exactly why I'm not in management at an organization that I don't own. It's when the people problem rots the management structure to the point that otherwise good managers can't be bothered policing the process monitors, and the lunatics start running the asylum.

There is no solution that I've found - short of leaving the organization, so I guess that's the next step. There is, of course, blatant disrespect of authority, and while that works short-term, it's not something that really is sustainable in the long-run. The trouble-maker can point out the problem, but then has to get back in line or he's a real danger to the authority of those in power.

I'll look back on this post in the future and point to this being a significant turning point in my career at The Shop.

I like Google Chrome. I use it everyday for all kinds of uses. But every now and then, they make a really silly change, and I find that it's just amazingly annoying, but there's little I can really do about it. A few versions ago, they added a drop-down in the tab bar for the logged-in username. This used to be a little image in the location bar, but they moved that out, and into the tab bar.

I know it's me - and it seems to me that the simple change would be to look and see how many profiles are defined for this guy, and if it's less than two, then don't show a thing - like it used to be.

I must be in such a minority that it's never thought necessary to fix. But it's ugly.

I'm looking at standing up an XML-RPC service written in PHP on the web, and I've been doing a lot of looking around, and found that Heroku really is about the best place for us at this point in time. It's got all the scalability we need, it's got Postgres, it's got redis, it's got all the things to mix-and-match all the work I need to do, and it's all in one place. I couldn't think of a better place.

But they haven't been doing PHP... until now.

This post talks about it, and it's really exceptionally simple... Start with a new directory and a simple index.php file:

  <?php
    echo "Hello World!";
  ?>

and in the same directory simply create a blank composer.json file:

  $ touch composer.json

Finally, create the git repo, commit the new files, create the heroku app, and deploy:

  $ git init
  Initialized empty Git repository in ~/hello_heroku_php/.git/
  $ git add .
  $ git commit -m "Initial import of Hello Heroku"
  [master (root-commit) 06ba0a7] Initial import of Hello Heroku
   2 files changed, 5 insertions(+)
   create mode 100644 composer.json
   create mode 100644 index.php
  $ heroku create 
  Creating safe-ridge-5356... done, stack is cedar
  http://safe-ridge-5356.herokuapp.com/ | git@heroku.com:safe-ridge-5356.git
  $ git push heroku master
  ...
  $ heroku open

This is amazingly simple! Just what I needed.

A while back I wrote about getting PostgreSQL support into PHP as it was packaged with Mac OS X 10.6. Since then, a lot has happened with Mac OS X, and PostgreSQL, and I find that I'm once again in need of developing PHP and PostgreSQL on my MacBook Pro. So I wanted to refresh this list and make it a little simpler at the same time.

Getting PostgreSQL 9.3

The first big difference in this post is that I'm switching to Homebrew and that's made the entire operation a lot smoother. There's no 32-bit/64-bit issue as Homebrew does both, and it builds in on your box, so again, a lovely customized solution with a simple command:

$ brew install postgresql

It even has a nice, simple informational blurb about how to start/stop and upgrade. Very nice. But now that it's on the box, and ready to roll, let's add in the PostgreSQL support to the PHP 5.4.24 that's installed with Mac OS X 10.9.

Activating PHP in Apache

The first thing to do is to activate PHP in the supplied Apache 2 with OS X 10.9. This is a single line in a single file - /etc/apache2/httpd.conf. There's a line and you need to uncomment it:

  LoadModule php5_module libexec/apache2/libphp5.so

and then add a new file called /etc/apache2/other/php5.conf and have it contain:

  <IfModule php5_module>
    AddType application/x-httpd-php .php
    AddType application/x-httpd-php-source .phps
 
    <IfModule dir_module>
      DirectoryIndex index.html index.php
    </IfModule>
  </IfModule>

which does all the other PHP configuration in a separate file to make upgrades easy.

Building in PostgreSQL extra for PHP

At this point we need to get the source code for the exact version of PHP that Apple ships. This is pretty easy by getting the PHP 5.4.24 source from a mirror and unpacking it into a directory. We then just run the following commands:

  $ cd php-5.4.24/ext/pgsql/
  $ phpize
  $ ./configure
  $ make

at the end of this process, you'll have a file: php-5.4.24/ext/pgsql/.libs/pgsql.so and that needs to be placed in the right directory and referenced in the php.ini file.

For Mac OS X 10.9.2, this is accomplished with:

  $ sudo cp .libs/pgsql.so /usr/lib/php/extensions/no-debug-non-zts-20100525/

and then edit the php.ini file to set the extension directory:

  extension_dir=/usr/lib/php/extensions/no-debug-non-zts-20100525/

and then add the line:

  extension=pgsql.so

Finishing Up

At this point, a simple restart of apache:

  $ sudo apachectl restart

and everything should be in order. Hit a URL that's a static file with the contents:

  <?php
    phpinfo();
  ?>

and you should see all the details about the PHP install - including the PostgreSQL section with the version of Postgres indicated. It's all up and running now.

I was having a pretty painful issue with SCP the other day and it took me a few hours to get to the bottom of it, but there's a lesson to be learned here. The problem was that I was able to SSH to a host, but was not able to SCP to that same host. The keys were good, the permissions on the key files were good - and SSH was just fine. It was just SCP.

Finally, I figure out it was my .bashrc file. I had added some functions in there, and they were doing fine with a terminal session, but the SCP "head-less" session was causing it to hang. Horribly. And that's the Safety Tip for the day:

Add this after the alias commands at the top of your .bashrc:

  # skip the rest if we're not interactive
  if [ -z "$PS1" ]; then
      return
  fi

and then you'll have the aliases if you need them, but you won't have the rest that could foul up the SCP session.

Very useful tip. Do it.