What's it all about, Alfie?

This morning I had the need of passing a completion function to a JavaScript function - I'm going to be doing an AJAX request, and when that request is done I want this function called. It's pretty obvious, really... when you're doing the request you might want a "Working..." message displayed, and when it's done, you want that removed. So here's the code I ended up with - pretty neat.

The calling function says:

  function showFitData(beg, end) {
    showFitsMessage('Working, please hold on...');
    var url = 'admin?cmd=fitData&out=json';
    // do the request and show the results
    doXMLRequestToTableWithFormat(url, fitTable, fitTableParams,
        function() { clearFitsMessage() });
  }

where we take the JavaScript function and encase it in another function definition. Pretty nice, but not as clean as I actually expected with JavaScript. Here nor there, this works and that's good enough.

The function called looks like this:

  function doXMLRequestToTableWithFormat(url, table, params, doneFcn) {
    var xhr = new XMLHttpRequest();
    xhr.open("GET", url);
    xhr.onreadystatechange = function() {
      // if we're not yet done, skip doing anything
      if (xhr.readyState != 4) {
        return;
      }
      // ...otherwise, get the tag for the answer
      var rawJSON = eval('(' + xhr.responseText + ')');
      var answerData = new google.visualization.DataTable(rawJSON);
      // let's get the 'negative as red' formatter for the columns
      var colorFmt = new google.visualization.ColorFormat();
      colorFmt.addRange(null, 0, 'red', null);
      // now let's process each numeric column, in turn
      for (var c = 0; c < answerData.getNumberOfColumns(); ++c) {
        if (answerData.getColumnType(c) == 'number') {
          colorFmt.format(answerData, c);
        }
      }
      // now we can draw it
      table.draw(answerData, params);
      // clean up what we no longer need
      delete answerData;
      delete rawJSON;
      delete colorFmt;
      // if we have a 'done' function, call it
      if ((doneFcn != undefined) && (doneFcn != null)) {
        doneFcn();
      }
  }

With this, it works like a charm. Nice to have worked this out. I'm sure I'll need it again and again.

Today I was checking on the latest Google Chrome for Mac and noticed that WordPress 2.9.2 was out and so I definitely needed to upgrade all my journals at HostMonster to the latest version. It's one of the most popular blog software packages, and as such, it's the target for a lot of malicious code. Better safe than sorry.

Today has been one of those days that I didn't see coming, and while there have been a lot of good parts to it, there have been a few that really make me want to head for the door and never look back. A long time ago a good friend said to me that one day he had a talk with his manager that used to be a really good coder. This manager said to my friend: "The toughest thing about management is learning to accept other people's code."

Amen, brother!

Today I was once again hit in the face with the reality that even when you give "marginal" coders examples, they're as likely not to even understand that these are the examples they are to follow. No... most of the time they'll google something and see a new library, or something, and read the magic words in their mind, and lock down on that as the solution.

Now it would have been easy to ask the other folks in the group - or the creator of the system: "Hey, I need to be able to do this... is there anything like that in the code?" And get the hint as to where to look. But even that is more than the "marginal" coder will do.

So I had these XML/URL-based data pollers in the code. They worked fine. Easy, clean, no problem. And yet this wasn't enough for a teammate to use. Nope. What we really needed was a Java-based crontab system. Yup. That's it.

Meanwhile, there's nothing these add that the other doesn't - other than code bloat, more dependencies, and a larger memory footprint. But what the heck? Why not, right? I'm sure there's no question of the "best decision" in this person's mind.

Meanwhile, I'm having to write more code to allow for some truly horrible data sources that are sending malformed XML, and poor socket handling on the part of a really flakey web server based on another technology (not Apache or Tomcat, that's for sure).

So once again, I realize that there's nothing I can do. Not one blessed thing. This teammate is not about to understand that what they've done is weaken the project. Less efficient, less compact, harder to understand. They just don't get it - and that's something I can't change, either.

I had a user that liked the pages I'd created in my web app, but they really needed to be able to export the data to Excel so they didn't have to transcribe it. Makes sense, and given that I know Excel takes CSV data, and the DataTable I wrote in Java has an option to output CSV, it should be easy.

Well... that was a good idea... but it really wasn't too hard once you get a few things cleared up. First, the CSV format. I had my CSV output method, toCSV(), formatting the numbers which was a terrible mistake. I needed to stop that. Then I needed to escape the CSV data according to these basic CSV rules:

• if the data has a comma in it - surround it with double-quotes
• if the data has a double-quote in it, double it, and then surround it with double-quotes

The first one is obvious. If I have data like this:

then the CSV needs to look like:

  32.5,Reggie,"Indianapolis, IN",5

and if I had data like this:

then the CSV needs to look like:

  32.5,"Reggie ""Dog"" Johnson","Indianapolis, IN",5

OK, these were new little wrinkles for me, but it didn't take too much time to make these fixes to my DataTable Java class. Now I have the right format, but how do I set up the rest of the response so I have Excel automatically handle it without a lot of grief?

Well... it turns out it's the header and content type. Each of my servlet requests has a parameter associated with them called out. If it's json, then the output of the request is JSON. If it's html, then it's HTML. This really helps in the debugging of the servlets as I can add &out=html... to the URL and see the results in the browser. Nice.

Well... let's make another 'type' of out - excel. Then, in my servlet I can do the following:

  String   out = request.getParameter("out");
  if (out == null) {
    out = "html";
  }
 
  if (out.equalsIgnoreCase("json") || out.equalsIgnoreCase("html")) {
    response.setContentType("text/html");
  } else if (out.equalsIgnoreCase("csv")) {
    response.setContentType("text/csv");
  } else if (out.equalsIgnoreCase("excel")) {
    response.setHeader("Content-disposition", "attachment;filename=\"myDump.csv\"");
    response.setContentType("application/vnd.ms-excel");
  }

I was lucky to get the header and content type from a few google searches, and then it was just a matter of putting it all together. Once I get my DataTable I can then ship it back to the caller with the code:

  if (out.equalsIgnoreCase("json")) {
    response.print(ans.toJSON());
  } else if (out.equalsIgnoreCase("csv") || out.equalsIgnoreCase("excel")) {
    response.print(ans.toCSV());
  } else {
    response.print("<center>" + ans.toHTML() + "</center>");
  }

When I put this all together, the client doesn't need to know a thing. The request comes in for the URL, and the data comes back, intercepted by Excel, and it fills in the cells. Pretty neat.

Today I knew that Marie was glad I was her Dad. She called this morning and had a problem with Pages '09 and an outline she had been writing as a homework assignment. She had accidentally compressed the outline into "one line per item" mode, and was in a bit of a panic because she needed to have it printed out for today.

I looked at what she was doing, then we went through all the menu bar settings and at the very right, there it was. Click that, and she was in business.

The relief in her voice was something I'll remember for a long time. Moments like this - where you can tell that one of your kids is really, honestly, grateful that they are your kid, are few and far between for teenagers. It was something I really enjoyed.

I've got some massive web pages - lots of AJAX requests feeding thousands of data points back to the client for inclusion into some Google Visualization widgets - it's as heavy-duty as many visualization apps I've created. But therein lies a problem - memory usage.

One of my users wants to have a dozen of these pages open and hammering away all day long, and while that's nice, it's not a good idea from the resource consumption angle. It would be ideal if there were a JavaScript event that told me when the view was completely hidden. But I've looked for that and there's nothing I can find that is going to tell me when to shut down the updating, or when to start it back up again.

So I need to do some memory profiling of the pages and try to figure out why they are getting out of hand. So I started digging into what Google Chrome had, and was pleased to see that it was really WebKit that had what I needed. Well... close, anyway.

Within Chrome, there are a lot of nice tools to show you what's happening. Not a lot of help when dealing with Google Visualization widgets as their code goes through an 'optimizer' and the objects are then pretty useless for help, but it tells me that the problem is most likely in their stuff and the Strings that are being created in the moving of data back and forth between the server.

None of this is really Rocket Science, but it's nice to have quick access to the tools to be able to see the difference in a few memory dumps, and even a little code profiling. Not bad at all.

Today I spent the entire day cleaning up a few things that I forgot about the data from the Hemlock database, and then writing the Java loader of that data so that it's all loaded when it's needed, updated on a decent interval (love that database polling) and then available in a reasonably decent way to the servlets that will need it.

Lots of heads down coding today.

Today was spent digging into the SQL of the Hemlock database and trying to find the data the users are asking for, then trying to organize it in the servlet so that it's possible to access what we need very quickly and with a minimum of fuss. Given the way these things are laid out, I was stunned that I got as far along today as I did.

I was able to get the stored procedures written to pull the data for today, and the last week (to give you a hint, I have to go to one database for one, and another database for the other). I was also able to get the product data and the month/strike data. Really... I was pretty impressed with what I was able to get done.

Heads down... it can pay off.

Today Gruber pointed out a link to an Open Source alternative to VLC. Given that VLC is hurting for the Mac developers, and that it's not able to even get a 64-bit version out the door, it seems likely that it might be heading out. If that's the case, then ripping DVDs is going to get very hard.

Movist seems to be nice, small, clean, and it works. Can't ask a lot more than that. I'll see if I need it, but it doesn't have the libdvdcss.2.dylib that HandBrake et. al. need to function. That's not a good sign.

Wonderful quote - horrible feeling if there's nothing to be gained. In this case, there wasn't. I had to once again dig into what I'm now calling Project Hemlock, the project I was given very early on at this job, and once again the lack of any documentation either written or comments in the code makes this a challenge I'd not wish to have.

There's something broken in a part of the Hemlock web app, and I need to fix it, but it's not a simple page - it's got business logic in the Flex client as well as the servlet gathering the data for the Flex client. It's not got a clear layout as to what it's doing or why, so the only way to follow the logic is to read each line of the code, understand what the variables are, why this might be happening, and then infer the function.

This logical single-stepping is very tiresome, and today I leave with very little done - more a point of a little more understanding, but still not enough to know where the problem is, or how to fix it. The only way I'm going to get this figured out is to pull in the original developer tomorrow, and have him tell me what's going on.

It's almost funny, but it's not. It's sad.