Building Up my C++ Variant Class

August 3rd, 2010

Today I spent quite a bit of time really fleshing out my variant class today. I needed to have a lot more functionality in the code as it was going to be an integral part of the ticker plant I'm working on. I needed to write a bunch of tests, and each new test uncovered either a compiler issue - like needing a new version of a method for a different use case, or a real bug in the code, which I had to fix.

Overall, it was a pretty good day, but it was all spent making the class a lot more useful to the developer that would be using it. Which, of course, is me.

Acorn 2.3.2 is Out

August 3rd, 2010

acorn152.jpg

This morning I saw that Acorn 2.3.2 was out with an impressive list of bug fixes and enhancements. This app has become the only real image editor I use, partly because my needs are pretty simple, and partly because this app is written for people that don't use Photoshop daily. It's easy to use, and that's the point. Excellent app.

Scrapped Boost Variant – Wrote My Own

August 2nd, 2010

Boost C++ Libraries

Today I messed around with the boost::variant problem I'd been dealing with lately trying to get the code to compile and work properly. Finally, after about five hours, I gave up. It's simply too hard to get working, and even if I did, the maintenance costs of dealing with these kinds of compiler errors would be far too high for a junior developer.

So I took a very different track: I simply wrote my own. Honestly, it wasn't all that hard. I took out the definition of the boost::variant, and in it's place I put a simple union:

  private:
    tVariantType      mType;
    union {
      std::map<std::string, variant>    *mMapValue;
      std::list<variant>                *mListValue;
      std::string                       *mStringValue;
      int64_t                           mIntValue;
      double                            mDoubleValue;
      uuid_t                            *mUUIDValue;
      bool                              mBoolValue;
      error_t                           *mErrorValue;
    };

and then all the setters cleared out the old value and replaced it with the new. It's something I've written before, and so I knew a lot of the pitfalls to avoid. But it's not as nice as a stack-based template version. When I change values I'm hitting the heap for new space. While this isn't horrible for a lot of applications, it kills performance when you're trying to do something really fast.

Still... this is far easier to understand, and once we have it all buttoned-up, there's no real chance of a leak, and it's a solid way to handle the variant problem.

Once I got the main part written, I was able to attack the serialization and de-serialization schemes for this guy - based on the work of another group who has defined the scheme we'll be using. It's decently flexible, and should be really nice to use across the board.

Still lots of testing to do, but I'll get to that tomorrow.

WordPress 3.0.1 is Out – Upgraded at HostMonster

August 2nd, 2010

This morning I saw that WordPress 3.0.1 was released, and SimpleScripts at HostMonster was telling me that it was time to upgrade. The upgrade took all of 5 min for all my journals, and when I was done, I was once again, safe for the time being. WordPress gets hacked a lot, and I try to stay up to date with it just because of that fact.

Google Chrome dev 6.0.472.14 is Out

August 2nd, 2010

This morning I noticed that Google Chrome dev 6.0.472.14 was released with the exact same release notes as 6.0.472.11 - "UI tweaks and no Flash loading". OK, I can see it's not a really exciting time in the Chrome group, but I wonder why they are doing all these releases and not fixing the Flash loading. Probably won't ever know...

Trying to Get Boost Variant Working

July 30th, 2010

Boost C++ Libraries

Once more unto the boost... this time to try and get the boost::variant working. In my particular application, I've got a self-defined data stream that can include:

  • Map - all keys are up to 256-character strings, values are variants
  • List - all elements are variants
  • Integer
  • Double
  • String
  • Boolean
  • NULL
  • UUID
  • Date
  • Error - which is a boost::tuple of a UUID, an integer, and a variant

and with the recursion in the definition with the map and list, I wanted to try the boost make_recursive_variant capabilities.

I sure do wish boost had better docs, because even getting to the point that the code compiles was an all-day affair. Primarily due to two lines of code:

  void variant::set( std::map<std::string, variant> & aMap )
  {
    mValue = aMap;
  }

and:

  void variant::set( std::list<variant> & aList )
  {
    mValue = aList;
  }

In theory, and in practice, this should set the value of the ivar mValue, the boost::variant, to the map and list, respectively. But I got the most insane compiler errors I've ever seen. Oddly enough, when I do:

  void variant::set( std::string & aValue )
  {
    mValue = aValue;
  }

everything works just fine. So it's clearly something about the recursive definition in the code. Possibly in the const-ness of one of something, but I tried all possible permutations I could think of. Nothing worked.

Finally, I tried this:

  void variant::set( std::map<std::string, variant> & aMap )
  {
    mValue.get< std::map<std::string, variant> > = aMap;
  }

and it compiled, but when I called this code and the value in the variant was not already a map, this threw a "bad cast" boost exception. Very understandable... I'm saying give me the existing map, and then set this guy there, but there's no existing map.

Exceptionally frustrating, but I'll have to hit it again on Monday.

Coda Notes is Out

July 29th, 2010

CodaNotes

One of the interesting demos at WWDC 2010 was in the Safari Extensions talk where Cabel of Panic fame showed something they hacked up called Coda Notes. It's a way to annotate a web page in Safari and then email it to your support staff, etc., and they can see all your annotations. This is super helpful for the web developer, and while I'm not sure I'll use it a lot, I certainly want to keep a handle on it. It looks just too amazingly powerful.

Awesome work.

Safari 5.0.1 is on Software Updates

July 29th, 2010

This morning I saw that Safari 5.0.1 is out on Software Updates and even though it's got just a few little security patches, it's nice to stay clear of the bad people out there bent on doing harm.

Google Chrome dev 6.0.472.11 is Out

July 29th, 2010

GoogleChrome.jpg

This morning I noticed that Google Chrome dev 6.0.472.11 was out with a very small set of release notes:

This release contains

  • UI tweaks and clean up
  • Additional stability fixes

Known Issues

I'm glad I'm not trying to use PDFs with Chrome. In any case, it's nice to stay up to date.

Amazing Use of calloc in The Magic Schoolbus

July 28th, 2010

Crazy Lemon the Coder

I ran across this today and I simply could not believe what I was seeing. It's right up there on Daily WTF - or should be, anyway. First, a little set-up...

This code is part of an incoming exchange data decoder. The Exchange will send messages on udp multicast, and it's up to us to grab them, decode them, place them in out message formats, and pass them on to all waiting listeners. What's important to realize is that these decoders are supposed to be efficient and fast. After all, they are decoding hundreds of thousands of messages a second. It's a lot of data.

So... the exchange dictates it's message format, and as in the olden days of the mainframe, most all the data is in fixed-length ASCII records. Specifically, the integer for the size of an order might be 8 characters and look like this:

  120.....

where the '.'s are spaces. Eight characters total, in ASCII format for the number 120. Simple. Not very efficient, but simple.

Since these fixed-length records will be end-to-end, there's no terminating NULL characters to make it easier to parse - you have to know what you're looking for. Well, the code I saw started with this method:

  /*
   * Convert unsafe char array string to int.
   */
  inline int uatoi(const char *nptr, size_t l)
  {
    int rv = 0;
    char *nullTermStr = cmalloc2(l + 1);
    if (nullTermStr == NULL) {
      errno = ENOMEM;
      return INT_MAX;
    }
 
    memcpy(nullTermStr, nptr, l);
    rv = atoi(nullTermStr);
    free(nullTermStr);
 
    return rv;
  }

where:

  /*
   * Malloc2 that returns a char pointer.
   */
  inline char *cmalloc2(size_t size)
  {
    return (char *) malloc2(size);
  }

and:

  /*
   * Malloc memory and initialize it to zero
   */
  inline void *malloc2(size_t size)
  {
    if (size < 0)
      return NULL;
 
    void *ptr = calloc(size, 1);
 
    return ptr;
  }

OK... this is really quite stunning. You want to parse a (char *) and so you duplicate it, by calling a useless method and then calloc with a repeat count of 1, parse it and then free the memory. And this is fast? For upwards of 10 fields a message, hundreds of thousands of times a second?

When I re-wrote the functionality I was decidedly simpler:

  /*
   * Convert unsafe char array string to int.
   */
  inline int uatoi( char *nptr, size_t width )
  {
    char    hold = nptr[width];
    nptr[width] = '\0';
    int     retval = atoi(nptr);
    nptr[width] = hold;
    return retval;
  }

Sure, I had to "loose" the const in the signature because I was modifying the data as I parsed it, but hey - it's a message from some data source - that's OK. It's also the same "logic" of using atoi() in the decoding. But now I'm not calling something to create some memory and then copying it, and destroying it. I can't believe they didn't look at all this before. It's incredible!

I know there comes a time when people don't look at the code anymore and just think the whole thing is too complicated... but really... guys... let's try a little harder. This is a horrible performance penalty for parsing. It should have been looked at long ago.

I guess I'm the one that decided to really look at it.