What's it all about, Alfie?

I have been working today to try and solve a problem that a new datasource to the main web app I've been working on since I arrived at the Shop. All the other data sources I've had to use sent me a complete, consistent, state in one packet. This meant that I was able to put the data into multiple database tables with foreign keys and have the 'acquired time' be a link from one table to another. This made it easy to see what arrived when. But the latest data source is more like an event-driven system, and it's not being so considerate.

This new guy sends packets of information at each level - and they are totally dissociated from one another. It's like they are destined for cells in a spreadsheet, where the user is supposed to know that the goal is the most recent data, even if it's not directly comparable and consistent.

For instance, there are about a dozen groups, and each group has from less than a dozen to upwards of 75 symbols in it. Each of these pieces of information is arriving at different times, and there's no guaranteed consistency between them. This includes the elements of the group and the group totals. It's meant to be "the best I know, with the least bandwidth". I can see the reason for the design, but it's very, very different from what we have for the remainder of the data sources. And this is causing me grief.

No longer can I look at the arrival time of the group totals, and know that this represents the arrival time of all the elements in the group. Nope... they have their own arrival times. So in order to get the complete state of this data source, the SQL to fetch it out has become far more complex, and time-consuming. It was really getting to be annoying.

And then a co-worker said something that made it all click. "Just buffer it up".

Wonderfully simple. I don't know why I didn't think of it before. Well... probably because I was thinking of processing the data as it arrived, and we can't do that. What we can do is to take the data as it arrives, buffer it saving the arrival time as the 'generation time', and then when the group totals packet arrives, use that as the 'acquired time', and save it as well as all it's components to the database.

This makes it possible to see the individual arrival times (generation time), as well as link the data all together with the 'acquired time'. It made the "shotgun" data source fit into the existing mold that I'd created for the other data sources. Sure it was simple, but it was something that hadn't occurred to be, and I was better for having listened to him and understood how to synchronize things up.

In the end, I took about an hour to fix things up and it is working wonderfully. Very neat.

For the last two days I've been busy just slugging it out in the trenches. Nothing fun, nothing exciting, just lots of drudge work that needed to be done, and I'm faster at doing it than anyone else, so I might as well just shut up, suck it up, get my head down and push through it.

Nothing exciting and glamorous about the last two days. Nothing.

For the last two days I've been hip-deep in a ton of little UI changes and additions to my web app. I can understand the reasons for the changes - it's the fist time they have actually sat down and thought about it for more than 2 seconds. When they did, and started to think about what they really wanted, they had some changes. Unfortunately, it was a lot harder to do this all a week (or more) after the initial creation of the page(s), but I suppose better late than never, eh?

There were things like re-arranging the columns in a table, and because of the nature of the Google Visualization Table, you can't re-arrange the columns in the view - they're fixed. So until they add this feature, I'm stuck putting the data in the right order before it gets to the client. Not horrible, but when it hits several pages, it's just a little time-consuming.

I also had requests for a few new fields. Nothing major, but putting it on four different pages meant a lot of little updates and making sure to try and pull together as much code as possible so that it's not copy-n-paste reuse. It's the easiest way to add the same thing in several places, but it would lead to a mess later on. I'll probably do a bit of work tomorrow to try and coalesce this even more. It's getting to be big.

In the end, I got the changes made and emails sent to clients so they knew the fields were there. Good enough.

Over the weekend, I stopped by the Apple Store in Naperville to pick up two of their small aluminum keyboards to use with two machines at work that used to share a keyboard/mouse with a switch-box, but last week were "split" so that I needed two keyboards in a pretty small desk.

For this desk, I need to get rid of the numeric keyboards to save the space. Just have to.

So the question is "How do I make it work with linux?"

Thankfully, xmodmap is going to do everything we need, and all I need to do is to get the keycodes from the keyboard and put them into the .Xmodmap file that I've already got to swap CapsLock and Ctrl.

Open up a terminal and type:

  $ xev

What you'll see is a new window with every X event getting printed to the standard out. Lots of information there, but in addition, you'll see every key press and release - and therein you can find the keycodes.

What I found was the following:

From this I was able to create the .Xmodmap that made the "Apple" keys act as "Alt" keys (their default appears to be to act as "Windows" keys) so it's easier to do what I need:

  !
  ! Swap Caps_Lock and Control_L
  !
  remove Lock = Caps_Lock
  remove Control = Control_L
  keysym Control_L = Caps_Lock
  keysym Caps_Lock = Control_L
  add Lock = Caps_Lock
  add Control = Control_L

  !
  ! Make the Apple keys Alt keys
  !
  keycode 115 = Alt_L Meta_L
  keycode 116 = Alt_R Meta_R
  add mod1 = Alt_L Meta_L Alt_R Meta_R

I may try to map some of the function keys to PgUp and PgDn keys, as those would be nice, but for now, it's enough this morning to get the Alt keys working on the new keyboards as that's how I switch between workspaces - which is essential.

I really like the additional desk space... that's a real plus. Just a little more fiddling with the location of things, and I should be settled in with the new hardware.

This morning I noticed that the Panic guys released Unison 2.0.3 with a short but important list of changes for this release.

I've had a chance to really put it through it's (reading) paces, and I have to agree with myself (a beta tester) that it's a really nice improvement over 1.x, but there's a few little cosmetic changes I'd make, and have mentioned to them in emails or bug reports. No show-stoppers, but still things I'd like to see changed.

This afternoon I ran into a problem between Java's URLEncoder and JavaScript's unescape() funtion. Ideally, I'd be able to create a URL in a Java servlet and ship it to the client and use JavaScript's unescape() function to parse the GET variables for the page. And it almost works. Almost.

The problem is the space character. The URLEncoder encodes the space into a plus sign ('+') and not the hex code (%20). The JavaScript unescape() function doesn't convert the plus sign into a space. So I have to do the following in my code:

  function parseGETVars() {
    var retval = [];
    var urlChunks = String(document.location).split('?');
    if (urlChunks[1]) {
      var pairs = urlChunks[1].split('&');
      for (var i = 0; i < pairs.length; ++i) {
        if (pairs[i]) {
          var parts = pairs[i].split('=');
          retval[parts[0]] = unescape(parts[1]).replace(/\+/,' ');
        }
      }
    }
  }

With the little change, I can use the Java URLEncoder and still get things working. Not too bad, but why on earth are these guys not doing the same conversions?

Lately I've been considering going back out on my own. Maybe with a few friends, but much, much smaller than what Port-to-Port became. When you have a lot of employees it changes what you're responsible for, and I want to stay doing things, as opposed to managing the business of those doing it. And the reason for this contemplation is that we have been discussing things at the Shop - things like vacation policy, sick days, and the like.

It all started when a co-worker brought up the fact that many places had no vacation policy. Actually, that's not entirely true. Their policy was that they didn't need a complex policy. Rather, they used the simple statement:

Achieving a balance between work and rest is important for everyone. Work has to get done and responsibilities covered. Good people know how to manage both.

and while this is heavily paraphrased, it's the essence of the policy that I liked.

Responsibility. Personal Responsibility, on a corporate scale.

I don't take a lot of vacation time, but I'd like not to even think about it. I work exceptionally hard and deliver results and products that are far above the norm. It's just the way it is. I'm not going to apologize for working 11 hr days, and working faster and smarter than most of the folks around me. Not all, mind you, and when I get to work with really sharp people, it's a treat. Really.

But it's the concept that I have to count these days. Moreover, when there is a "pool" of vacation and sick days, most people will look at sick days and say "I'm not that sick today to use a vacation day" - and so they bring their germs to work. Thus putting all of us with families at risk. It's the culture that there's a limited resource (days off), and the use of them needs to be personally justified.

This can get so bad that when there are "use it or loose it" days, people take off when they don't really want to because they feel they have to use this resource. Silly, yes. But very real. I've seen it happen at the Shop and I've only been here nine months.

As this email 'conversation' developed, the Netflix corporate culture docs (recently released) came up. And they are amazing. It's been talked about quite a bit, and I have to agree with all of them. What gets me most today are these:

• Adequate performers get generous severance packages.
• Evaluate performance based on effectiveness and results.

They go on to say that they see themselves as a Major League baseball team. Pay well, treat well, but expect stellar performance. If not, then be gracious, and show them the door. It's perfect.

Recently, I've been struggling with the very slip-shod work of people not really paying attention. It's not that they are really doing their best and just aren't capable of doing better - it's that they are being careless because they think it doesn't matter. They are wrong. It all matters.

So while I don't hold out a lot of hope that things will change at the Shop, I can hold out hope that I'll find the magic ingredients to cut out on my own, or with a few friends, and set up some place that adheres to these same guidelines. It's about the only new job I'm interested in looking at.

This morning I noticed that CoRD 0.5.2 was out, and with it, a hope that I may never have to have another Windows box again. This is an incredible app - works every bit as well as Windows' own Remote Desktop client for everything I need: get to a machine, run what I need, fiddle with files and settings, and get out. It's amazing.

With the update it's running in 64-bit on Snow Leopard, and several crashes have been fixed. It's also got the self-install/on-disk-location checks which have been talked about a lot in the web lately for Mac apps. I'm encouraged when I see this kind of thing picked up by really nice open source projects. Clearly, these guys are good, and that's great news.

I'll keep using this as long as it's available because I never want to have another Windows box, and there's no way business is going to give them up.

This morning I noticed that the Mozilla group has finally released Firefox 3.6 - with a stunning list of changes. Among the biggies for me:

• Protection from out-of-date plugins to keep users safer as they browse.
• Improved JavaScript performance, overall browser responsiveness, and startup time.
• Support for new CSS attributes such as gradients, background sizing, and pointer events.
• Support for new DOM and HTML5 specifications including the Drag & Drop API and the File API, which allow for more interactive web pages.

but there's a lot more there for everyone.

When there's a major update to Firefox, like this, all the themes and plugins fail until the developers have a chance to update them to the new system. This morning was no exception for me. I had to mess with quite a few things in order to get Firefox working on Mac, Windows, and Linux.

Theme Problems - Whitehart 3.6.5 [XP & Linux]

I use the Whitehart theme for XP and Linux and of course it's not ready. Sadly, there's nothing I can do here as it's not in my skill-set to mess with the theme. I'll just have to live with the default theme until they get around to updating it.

XML Rendering Problems [XP & Linux]

I need to look at the XML output of several different servlets and because the servlet's content-type is text/html as opposed to application/xhtml+xml, I don't really see the XML - just some of the unbounded data in the XML. This isn't very useful. The old plug-in I used to force a content-type switch wasn't compatible with Firefox 3.6, so I tried Force Content-Type 1.2.2. What a gem!

I simply install this, activate it, and then put in the mapping for the URLs I need using JavaScript regex and BINGO! It works like a champ! I spent nearly half a day initially trying to figure out what the problem was, but this time, this guy had me up and running in less than 15 mins. That's fast.

If you need to see XML in Firefox on XP or Linux get it.

Java Plug-in [Linux]

This was a puzzler until I started googling it... the Java plug-in for Linux for Firefox 3.6 changed from the old libjavaplugin_oji.so to the new libnpjp2.so. For the JDKs distributed from Sun, this file is in the same basic directory structure, and a simple find will find it. But there's no errors in using the old one, it just doesn't load. Switch to the new one and restart Firefox, and all is well.

This little tidbit wasn't in any of the docs I read, and I wasn't the only one that thought this was an oversight. I can't imagine that after years of using the old plugin, I'd have thought to switch on my own. I'm sure glad one of the developers spoke up on this thread. Yikes!

I've been working pretty hard today - and for me that's saying something. There were a few things that were very near to completion, and I wanted to get them all out to the users today so I could write them off my list. I had a deployment to do, then I needed to incorporate a few more data sources to my web app, then I needed to update a few pages with additional features the users had asked for. Nothing took more than an hour or so, but there were a lot of those hour-long projects I wanted to get through.

By the time early afternoon rolled around I found that I was in a pretty crummy mood. I started to think about a conversation we had this past week about my need for appreciative feedback. She had suggested that I volunteer at the Apple Store (or get paid, either way) because the people coming in there are (usually) very appreciative of the Genius Bar's ability to fix things up.

I didn't have the heart to tell her about the bad stories I'd heard from the Genius Bar workers on the net, but that's not the point. She was saying that the work I do is nice, and it's fulfilling, but everyone wants a little positive feedback for their efforts. Otherwise, it's going to lead to burn-out.

As I was thinking of this I realized she was totally right. What's more, while I may have felt that my last job was far more hostile, it was also far more appreciative.

The harsh remarks of my last place were, at times, clearly over the top. But primarily confined to a few folks. It was certainly not universal. And when they came, it was clear they were issued by a person that was very upset about something - maybe the problem they were talking about, and maybe it was something else entirely. But in every case, I was able to clearly identify the issues involved and indicate if there was something that I could do to resolve the issue.

No question, there were folks there that could go over the top with verbal anger at times.

What I hadn't realized was that there were other folks - the majority of folks, in fact, that would go over the top with appreciation.

The majority of the support folks... the majority of the operations staff... even the majority of the traders were very appreciative of my work. True, I had been there many years and built tools that were the cornerstone of the risk analysis, but still... there were web systems I built, and legacy apps I maintained and all those people were appreciative of the efforts.

Even the loudest of the 'loud angry' folks was very appreciative at times.

Which brings me to why I feel so crummy this afternoon - there's virtually no appreciation in this position now. There's no "loud angry" people either, but if I had to accept the latter to get the former, I think I'd do it every day of the week. If I'm going to be working 11 hr days, and making a 90 min commute - each way, I'd like to know that what I do matters.

In the last four years of my old place, I almost never got yelled at. Plenty of times in the first three, but less and less as time went on. I was good at what I did, and when I said I'd get it done, it got done. No one had a reason to get angry at me. My stuff just worked. So while I was working in a hostile environment, I wasn't the target of much hostility. I was, however, the recipient of a good bit of the appreciation.

I miss that. I really do.

If I'm going to work for no appreciation, I'll still do it. I'll just pull up quite a bit short of where I've been working because there's no one giving back to me. Also, the faux appreciation of some folks doesn't count. You can tell when it's genuine, and when it's "just words".

So today has been an important realization for me. I realized that if I'm going to be leaving it on the track every day, I'm going to need to hear a genuine "good job" now and then. If I'm not going to get it, I can't really leave it all on the track -- I'll burn out. And I've been very close to burn-out the last few weeks.

So it's time to work as hard as I can - for myself. If I get more from the users, that's great, and I'll be able to give a little more, but if not -- and they are by no means required to do so, then I'll at least have enough left over for the family in the evening.

And I know they deserve it.