What's it all about, Alfie?

Looks like we have a new version of Google Chrome dev this morning - 23.0.1271.10 with a sparse, but informative list of release notes. With this update, it appears they are about to jump to 24.x - but we'll have to wait and see. They may all be working on getting iOS Maps out 🙂

In any case, fixes for Flash are always good (I really dislike the implementation of the Flash interpreters I've seen from Adobe), and while I'm not a Windows 8 fan, it's nice to throw them a bone once and a while.

Today I did a little digging on the idea of exporting a Google Visualization Table to CSV all in javascript. Face it - the table is already there… it's got the data… the trick is how to get it all up and going for the CSV export. Well… as it turns out, it's not all that hard. I was pretty surprised.

The core of it is really the Google Visualization DataTable. Since that's the core of most of the Visualizations, that's a great universal starting point. What we're really doing in the code is making a simple javascript method that will make a URI and encode it, such that when it's opened, it'll appear as a download to the browser and be kept as a file.

The first part is to save the DataTable when you render the Google Table on the page:

  // this is the Google DataTable we'll be creating each time
  var dtable = null;
 
  // This method looks at the selected data set and loads that into
  // a new table for the target div and redraws it.
  function render(tbl) {
    // save this data table for later
    dtable = tbl;
    // now create a Google Table and populate it with this data
    var dest = document.getElementById('table_div');
    var table = new google.visualization.Table(dest);
    table.draw(tbl, table_config);
  }

At this point, we have the DataTable, and then we can place the button anywhere on the page, I happened to place it, centered at the bottom of the page:

  <p align="center">
    <input type="button" id="toCSV" value="Click to download data as CSV"
     onclick="toCSV()" />
  </p>

So that when the user clicks on the button the following code will be run:

  // this downloads the current data table as a CSV file to the client
  function toCSV() {
    var data = dtable;
    var csvData = [];
    var tmpArr = [];
    var tmpStr = '';
    for (var i = 0; i < data.getNumberOfColumns(); i++) {
      // replace double-quotes with double-double quotes for CSV compatibility
      tmpStr = data.getColumnLabel(i).replace(/"/g, '""');
      tmpArr.push('"' + tmpStr + '"');
    }
    csvData.push(tmpArr);
    for (var i = 0; i < data.getNumberOfRows(); i++) {
      tmpArr = [];
      for (var j = 0; j < data.getNumberOfColumns(); j++) {
        switch(data.getColumnType(j)) {
          case 'string':
            // replace double-quotes with double-double quotes for CSV compat
            tmpStr = data.getValue(i, j).replace(/"/g, '""');
            tmpArr.push('"' + tmpStr + '"');
            break;
          case 'number':
            tmpArr.push(data.getValue(i, j));
            break;
          case 'boolean':
            tmpArr.push((data.getValue(i, j)) ? 'True' : 'False');
            break;
          case 'date':
            // decide what to do here, as there is no universal date format
            break;
          case 'datetime':
            // decide what to do here, as there is no universal date format
            break;
          case 'timeofday':
            // decide what to do here, as there is no universal date format
            break;
          default:
            // should never trigger
        }
      }
      csvData.push(tmpArr.join(','));
    }
    var output = csvData.join('\n');
    var uri = 'data:application/csv;charset=UTF-8,' + encodeURIComponent(output);
    window.open(uri);
  }

You can see the entire page here:

The downside of this is that the file will have an unusual name. On Mac OS X with Safari 6.0.1, it's "Unknown". On other platforms, I'm sure it's something nearly as odd and useless, but that's the name of the game. There's seemingly no way to get the name of the file in the URI or the window.open() method.

Still… I'm pretty pleased. We're looking at a 100% client-side, javascript solution to the CSV generation problem. That's pretty nice. If you look at the code, there's really very little that's exclusive to the Google DataTable - it's really just the means to get the headers, and the row and column data. We could have easily built this from any regular data source and made that work as well.

Sweet.

I was looking at the results of the overnight runs and realized that I really needed to work making the nightly runs far more robust by retrying the complete division run if there were errors in the processing. The trick was - How?

So I came up with the idea that the summary script that's gripping the log of the process, could, in fact, look at the log and determine if the run was a "success". Then, I could have it return a simple error code and the nightly run script could detect that and place the division into a retry queue if it failed.

Great idea, but getting the Bash scripts to do it was a little more work than I'd expected. Oh sure, the exit value from the summary script was easy, I just had to pick the right thing to test in that script, and after a few tries, I got what I needed. No… this was about the dealing with the string vs. integer nature of the bash variables.

For example, this is OK:

  if [ $pass -gt 3 ]; then
    break
  fi

unless $pass is a string, and then you get an error and die. So it's possible to say:

  if [[ $pass > 3 ]]; then
    break
  fi

but this seems to cause me grief:

  if [[ $pass >= 3 ]]; then
    break
  fi

It's not easy to figure out all the edge cases until you've written more than a few bash scripts using a feature, but hey… that's the power and grief of using bash.

In the end, it was really much easier to force the type of variables and make sure they are OK, than it was to try and figure out why 'greater than' was OK, but 'greater than or equal to' wasn't. The goal was to have a couple of scripts tied together that allowed me to ensure that they nightly runs worked, and worked well, and I've got that now.

Problem solved.

There are several problems related to the underlying sockets in JRuby that are fixed in 1.7.0, so The Team was interested in moving to it as soon as realistically possible, and today was that day. Everything seemed to be working just fine, but then when we tried to run the code in the CI environment we got all kinds of failures due to the code we were using. While CI was running JDK 1.7, and I think we're all on JDK 1.6 for our laptops, the glibc that exists with CentOS 5 is too old to run the JRuby 1.7.0 code.

I have used CentOS 5 for a long time. It's stable, but it's old. There's a lot to be said for stable, but when it's impossible to deploy things because the fundamental libraries are too old, or the packages are too old to run a recent package from someone else, then it's time to really consider a change.

Here, the solution seems to be to build a custom glibc and then just force the LD_LIBRARY_PATH for the particular application. But that's got all kinds of stability problems in it. Glibc isn't like just another library - it's a fundamental library in the OS. There's tie-ins to the compiler and a lot of other things that aren't as easy to assume will be "OK" with just a new version of glib.

But I've already asked, and it's just not going to be possible to update to a more recent OS. Politics and support make this just right out.

Sad… because those fixes would be really nice.

This morning I noticed that Google Chrome dev 23.0.1270.0 was out, and there's a nice set of changes for this release. There's the V8 javascript engine - 3.13.7.1, and then there's quite a bit of codec/playback work done as well. I don't typically do a lot of that, but I can see that others do, so it's a good thing to get nailed down.

The speed is nice, the redrawing is superb… very nice tool.

This morning I got a note about a Sublime Text 2 plugin for working with Salesforce.com APEX code as well as Git integration - MavensMate-SublimeText. This is a pretty amazing project for working with Salesforce.com code. While I haven't installed it on my laptop, it looks to be the thing if you're doing any significant amount of Salesforce.com coding. There are pages for running tests as well as immediate mode execution, and then of course, the highlighting and 'Code Assist'.

Very interesting. If I get into some real Salesforce.com code, I'll have to look harder at this.

This morning I noticed that again, Google Chrome dev is bumped to 23.0.1262.0 with some more good release notes. There's an update to WebKit (537.10), and the V8 javascript engine (3.13.6.0), and at least one Mac-specific fix. Nice! The page refresh speed is really quite amazing, and has been for the last two releases. It's really impressive. I'm hoping they keep it up!

This morning a co-worker tweeted that he found this spiffy Bash prompt generator built in Python. Now I'm not normally one to adorn my shells with command prompts like this, but Wow! this is impressive. I mean it's got your name, the path, the git branch you're on, and even history with a red background in case of an error. That's pretty impressive.

Then again, it's probably a serious Python script that takes some time to run, but for those that want a pretty prompt, this looks pretty amazingly stylish. I gotta hand it to him. It's really close to something that's nice enough for me to start using it.

So it goes… I'll keep it in mind for now.

I had to do a special gem today, and I wanted to get down how it was build and deployed. This is because I didn't remember it and I'd already done this once. So here goes.

First off, make sure that you have the skeleton of a gem - including the gemspec file - it comes with all GitHub repos, and if you make a new gem with the Bundler, it gives you a simple skeleton as well.

Next, write the code and then build it:

  $ rake build

this should deposit it in your pkg directory.

Upload it to a site - like ruby gems. Simple.

It's worth noting that it might help to delete older versions of the gem. This is easily done with:

  $ gem uninstall couchrest

If there are multiple versions of the gem installed, it'll give you a choice. If there's only one, it's gone.

Fix your Gemfile and then reload all the gems with:

  $ bundle install

Today I was really battling a nasty problem with the CouchRest client for CouchDB. This is a ruby gem, and in general, it's pretty decent, but it really starts to fall down when you need to use a proxy to get to CouchDB, and this guy starts having all kings of problems.

There were timeout issues, so I decided to try and make it work. So I forked the project on GitHub, and started to get to work. THe major point was that the underlying RestClient gem had the ability to set things like the timeout for creating the connection as well as timeouts for reading, etc. It's really very flexible. My goal was to allow the settings to be applied on a per-database basis. Then, for every command, use these as the defaults, but overlay any call-time options as well.

The idea was really nice. I was planning on submitting a pull request for this as it only took me about an hour to do. But when I went to test it, it failed with some 502 Bad Gateway error.

Argh!

More proxy problems!

Then I was talking to one of the guys in the group about this issue and he brought up that I could write to my local CouchDB, and then replicate it to a different database on a different server!

BING!

This is exactly what I'd been looking for. I get fast and efficient writes to my CouchDB, but it gets written up to the shared server as I'm connected to the network. This is great!

The configuration is simple - it's a simple doc in the _replicator database, and I'm in business. This is really quite amazing. First, go to the overview in the CouchDB web page, and select the _replicator database:

then create a new document:

Finally, populate it with these essential key/value pairs:

• source - this is the source for replication - it only goes one-way, so this can be a local database, or a remote one. But it's where the data is coming from
• target - this is the destination for replication - again, local or remote, it makes no difference. Make sure to put in the port and the database name in the URL
• proxy - if needed, put in the URL of the proxy to get to either of these databases
• continuous - set to true if you want it to always be replicating

Save this document and then look at the CouchDB status page to see it replicate. It's like magic! OK, not really, but the handling of the proxy is so far superior to the way that CouchRest was dealing with it it's not even funny. This just works!

I'm more and more convinced that CouchDB is an amazing product.