I’ve Seen How the Sausage is Made

September 16th, 2010

pirate.jpg

Spam. Social Engineering. Everything old is new again.

I've received a lot of new spam today - all with HTML attachments that I'm supposed to be dumb enough to actually click on. They are all about one thing or another. I'm either getting money, or it's something about the softball team, but all try to look like something I'd get - but they all fail to realize that I've seen how this all works, and I'm not about to click on anything I don't know the source of.

What continues to amaze me is the fact that they send this at all. They must get people clicking on them because they send them. So many times...

I just wish I could put my name on some hacker's list that says "Don't even bother...", but then they'd probably just make that the one list they'd try harder on.

Sigh...

Helping Folks Can Really Slow You Down

September 15th, 2010

cubeLifeView.gif

I've been given the opportunity to see if a developer in the Shop would be a better fit for the project I'm working on than the one he's currently on. Well... sure. I think to myself - why not? Well... the answer to that question is, of course, that adding this person to the group would slow me down to the point that I miss every deadline I've promised, and drive me to the brink of sanity.

I'm not sure what I'd call this personality, but it's an interesting mix. There's skill there to be sure. But getting that skill out is a very time-consuming operation. I'm not sure it's really worth it, as the work seems to be something I end up re-writing anyway, but it's based on a decent set of skills. Just bad ideas, I guess.

So I'm working with him to try and see if I can correct the problems - thinking they aren't that severe. After all, if they got here, the theory is they are valuable, right? So they are worth saving, right? At least those are the assumptions I'm working under.

So I'm asking him to have a look at some unit tests, and write a few more. This is how I like to introduce new developers to the codebase. It gives them an idea of how the code looks, how the tests are built, and how to compile and run the code on a very baby steps approach. It's been a pretty good approach for me so far.

For this guy it didn't seem to help a lot. I still had to do a lot of re-writing. More importantly, the designs, which should have been simple and small extensions to the existing objects, were instead complex, tangled messes of code. I don't think I've ever seen as bad an extension to a class as I've seen here. He just didn't even seem to try to look at the existing code and make his like it in any way.

But while this is all understandable, and I can re-write anything, the big surprise to me is the time it takes to talk to this person. It's measured in hours. Really. It's not like this is someone that can take an idea or two and run with it. Nope. It's got to be argued over and over again - and then, he'll do something about it. The code is bad, yes, but the time sink is really what pains me.

I'm throwing away large chunks of my day to this person, and I'm not sure any of it is doing any good. It's as if there's nothing being really transferred from me to him. He's not learning from the corrections I'm making to his code, and he's not learning from the talks we're having. I've had talks with people in his previous group here at The Shop, and they are tickled pink that he's with me now. They are guessing it saves them three hours a day to have this person off their team.

I'm being a good corporate citizen and doing what I'm asked, but I've already alerted my manager and the project lead that this person is going to double all my time estimates. I don't know what they are going to do, but until they tell me something, he's my problem and I need to come up with some way to at least keep the conversations to a minimum.

Yikes. I had no idea what I was getting into.

Google Chrome dev 7.0.517.5 is Out

September 15th, 2010

GoogleChrome.jpg

This morning I checked and Google Chrome dev 7.0.517.5 was out so I updated to see what the new features were. I have to say, it's nice to see that they are getting more Mac-like, and I hope it's a good sign. There has been a lot of grief in the press lately with Google and Apple, and while I'm not trying to say everyone should be nice and happy, it's really much nicer for me if the two companies I use a lot would be at least professionally polite. It makes for happier folks all around.

Anyway... Chrome is updated and that's always a good thing.

Transmit 4.1.1 is Out

September 14th, 2010

Transmit 4

I got a tweet today from the great guys at Panic that they had released Transmit 4.1.1 after a fitful release of 4.1, which had a bug with the Sparkle updating. The 4.1.1 release has an impressive list of features including a massive update to the Transmit Disk feature. It's now independent of MacFUSE, so I removed that from System Preferences, and it works with 64-bit kernels. It always was a good to great product, it's just getting better and better for what I need. Amazing application.

Debugging Some Nasty Problems in Pair Programming Style

September 14th, 2010

This afternoon I had the most horrid debugging session in recent memory. It was really just that bad. The problem was that the code the new guy was writing was giving segmentation faults when my code wasn't. Additionally, there seemed to be a serious data problem in the information we were getting from another group's code. Not fun.

The first problem I ran into was the formatting of a timestamp into a human-readable string. The original code was:

  std::string formatTimestamp( uint64_t aTime )
  {
    char   buf[32];
    // see if it's w.r.t. epoch or today - we'll format accordingly
    if (aTime > 86400000) {
      // this is since epoch
      time_t  msec = aTime;
      struct tm   when;
      localtime(&msec, &when);
      // now make the msec since epoch for the broken out time
      msec = mktime(&when);
      // now let's make a pretty representation of those parts
      snprintf(buf, 31, "%04d-%02d-%02d %02d:%02d:%02d.%03d",
               when.tm_year, when.tm_mon, when.tm_mday,
               when.tm_hour, when.tm_min, when.tm_sec,
               (int)(aTime - msec));
    } else {
      // this is since midnight - let's break it down...
      uint32_t    t = aTime;
      uint8_t     hrs = t/3600000;
      t -= hrs*3600000;
      uint8_t     min = t/60000;
      t -= min*60000;
      uint8_t     sec = t/1000;
      t -= sec*1000;
      // now let's make a pretty representation of those parts
      snprintf(buf, 31, "%02d:%02d:%02d.%03d", hrs, min, sec, t);
    }
    // ...and return a nice std::string of it
    return std::string(buf);
  }

My problem typically is that I test some things, but not all edge cases. In this case, I hadn't really tested the "since epoch" code, for there's a few doozies in that section.

First, localtime_r() is seconds since epoch, not milliseconds, like I'm assuming. Duh. That means we need to fix that up:

  std::string formatTimestamp( uint64_t aTime )
  {
    char   buf[32];
    // see if it's w.r.t. epoch or today - we'll format accordingly
    if (aTime > 86400000) {
      // this is since epoch
      time_t  sec = aTime;
      struct tm   when;
      localtime(&sec, &when);
      // now make the sec since epoch for the broken out time
      sec = mktime(&when);
      // now let's make a pretty representation of those parts
      snprintf(buf, 31, "%04d-%02d-%02d %02d:%02d:%02d.%03d",
               when.tm_year, when.tm_mon, when.tm_mday,
               when.tm_hour, when.tm_min, when.tm_sec,
               (int)(aTime - sec*1000));
    } else {
      // this is since midnight - let's break it down...
      uint32_t    t = aTime;
      uint8_t     hrs = t/3600000;
      t -= hrs*3600000;
      uint8_t     min = t/60000;
      t -= min*60000;
      uint8_t     sec = t/1000;
      t -= sec*1000;
      // now let's make a pretty representation of those parts
      snprintf(buf, 31, "%02d:%02d:%02d.%03d", hrs, min, sec, t);
    }
    // ...and return a nice std::string of it
    return std::string(buf);
  }

The next one is about the broken out time struct. The year is offset by 1900, and the month offset by 0, so I had to fix those up:

  std::string formatTimestamp( uint64_t aTime )
  {
    char   buf[32];
    // see if it's w.r.t. epoch or today - we'll format accordingly
    if (aTime > 86400000) {
      // this is since epoch
      time_t  sec = aTime;
      struct tm   when;
      localtime(&sec, &when);
      // now make the sec since epoch for the broken out time
      sec = mktime(&when);
      // now let's make a pretty representation of those parts
      snprintf(buf, 31, "%04d-%02d-%02d %02d:%02d:%02d.%03d",
               when.tm_year+1900, when.tm_mon+1, when.tm_mday,
               when.tm_hour, when.tm_min, when.tm_sec,
               (int)(aTime - sec*1000));
    } else {
      // this is since midnight - let's break it down...
      uint32_t    t = aTime;
      uint8_t     hrs = t/3600000;
      t -= hrs*3600000;
      uint8_t     min = t/60000;
      t -= min*60000;
      uint8_t     sec = t/1000;
      t -= sec*1000;
      // now let's make a pretty representation of those parts
      snprintf(buf, 31, "%02d:%02d:%02d.%03d", hrs, min, sec, t);
    }
    // ...and return a nice std::string of it
    return std::string(buf);
  }

Here, now, we finally have something that's right. The second big issue was with the encoding of integers with Google's varint system. In general, it's vey effective, but I was seeing the timestamps coming from a service as twice what they should be. Made no sense whatsoever - until I saw the code.

In order to efficiently compress signed integers, Google came up with the idea of zig-zag encoding. Basically, you sort the integers by their absolute value, negatives first, and then pick the row in the sort as the coded value. Something like this:

Value Code
0 0
-1 1
1 2
-2 3
2 4

From this table, it's easy to see that if you expect to see an encoded unsigned int, as I did, the signed int can look to be twice the size! When I saw they were using signed values where I was expecting an unsigned value, I knew it was the zig-zag encoding and fixed that right up.

These should have taken me about 30 mins to find. As it was, they took hours. Why? The developer that's new to the team wanted to "watch". Now by "watch" they meant "watch a little, but talk a lot", and there's the rub. When I'm debugging, I need quiet. I need to see what's really going on, why the code isn't acting as I expect it, and what assumptions and ideas I had that were wrong. Having a Chatty Kathy sitting next to me was, to say the least, unhelpful. To be accurate, a hinderance.

If I felt they learned anything in the process, I'd settle for that as a positive outcome. But I'm not at all sure they learned a thing. I think they just don't like working alone, and so saw this as an opportunity to have a "working buddy". Well... today is the last day for that. Never again.

Gotta nip this in the bud.

Cleaning Up Someone Else’s Code

September 14th, 2010

This afternoon I've spent a lot of time cleaning up someone else's code. This person is new to the group, and I've been asked to see if he will fit into the project. I do have my doubts...

I had him put in a few tests and write a few methods on an existing class. This wasn't meant to be hard, it was meant to be easy, but it turned out to be something that I needed to be more explicit about.

The code was an interesting combination of decent STL C++ code, but completely optimistic, and no comments to speak of. Those comments that were there weren't even complete sentences. While I'm not an english major, I think the comments are the only real help that an author gives to the person trying to figure out what they did. Many think of this as a weakness, but I know better. I'm often the one that reads what I wrote 6 to 9 months after the fact, and I'd like to cut my future self a little slack.

So I had to not only re-arrange things, and comment them, I also had to remove the optimism of the original code, and in many cases, improve the speed as the use of some STL ideas is fine if speed isn't an issue, but in a ticker plant, that's just simply not the case.

What I have come to realize is that the new guy is not all that great a coder. With all the examples they had to work with, they wrote code that was completely out of context with the rest of the class. This shows a real lack of awareness to me. We'll see how he progresses in the coming days.

Skitch Beta 8.7 (v2716) is Out

September 14th, 2010

Skitch.jpg

Skitch 8.7 - the final Beta is out and they promise that very soon they will have the 1.0 of the app and the website out with a dramatically new look. I really think they'd have to charge and arm and a leg for me not to pay it for this app - it's just that good. The only difference seems to be the extension of the timeout, but hey, at least it keeps working.

Good news, and something to look forward to.

MarsEdit 3.1 is Out

September 14th, 2010

MarsEdit 3

I got a tweet this afternoon about an update of MarsEdit 3.1. It includes 64-bit support as well as Lightroom 3 compatibility. Looks nice. The whole change list is:

MarsEdit 3.1

  • 64-bit Compatible
  • New per-blog setting to constrain image media to pre-set size
  • Support for Lightroom 3 media libraries
  • Added new "Paste HTML Source" command for e.g. embedding YouTube easily in rich mode
  • Enabled plugins for rich editor so you can e.g. see embedded movies
  • "Send To Weblog" command now works when preview window is front-most
  • Optional support for Google Data API 2.0 with Blogger
  • Bug Fixes
    • Fix issue that prevented ejection of external media volumes
    • Fix some cases where the preview window could turn blank
    • Make sure changes to the preview template save to disk immediately
    • Restore functionality of "send post to blog" scripting command

It'll be interesting to see if the 64-bit change makes it a little more responsive at times. That's about my only complaint. It'll get there, and it's a great app.

[9/15] UPDATE: seems there was a single, nasty bug in 3.1, and so they released 3.1.1. Seems to be a lot of that going around these days (Transmit).

Cleaning Up Ticker Plant Client

September 14th, 2010

This morning I've been cleaning up my ticker plant client - making the ZeroMQ listener of messages smart enough so that when the last listener is removed, the ZMQ socket is torn down thus shutting off the reliable multicast stream of messages that no one is listening for any more. It's a bunch of little details like that - things that will make the code nicer, but not really any less functional for the initial testing and rollout.

It's the "second pass" on the code - adding all those touches you'd expect to see, but aren't required in the first workable cut of the code.

Finished Adding Conflation to all Necessary Endpoints

September 10th, 2010

GeneralDev.jpg

I'm relieved to know that I have all the conflation queues and good byte-level queues in all the necessary endpoints in my communications library. The ZMQ receiver and transmitter have them, as to the TCP client and client proxy. This is pretty much all that's needed, and adding them where they aren't necessary is loading the system with threads that are processing data as fast as possible. It gets inefficient.

But for now, I have them in and things are looking and running well. One step closer to a full-up test. Now I need to start pulling these together into more complex components and then testing box-to-box performance. It's getting there.