What's it all about, Alfie?

Today, in addition to a few meetings, I've been hard at work trying to work all the messages from one exchange data feed into my new ticker plant. The previous version of this project had over 300 messages - one per exchange message. I'm going for a far more minimalist design: if it's not a price or a trade, it's in a free-form variant message, and that's going to really help me in minimizing the number of messages people have to understand and deal with.

Yet there are always twists.

Today I realized that there are messages from the exchange that don't update their sequence number. OK, maybe they aren't critical to the function of the ticker plant, but I didn't want to throw them out at the UDP receiver. So I had to adjust the methods on my data source to allow for ignoring the sequence number checks. It made things a little more complex, but in the end, it'll be a better system.

That's where I'm at these days - taking what I think is a good idea and applying a real problem to it and seeing where it needs to stretch and fit. I'm hoping that I can get this all done in a few more days and then get to the real-world tests and verify that things are working as desired.

I've got my fingers crossed...

The relentless advance of Google Chrome has this morning deposited 6.0.472.22 on my little digital doorstep. The release notes are pretty sparse:

All Platforms

• UI Updates
• Stability Fixes
• (Issue 49493) Fix some problems with SSL connections through HTTP proxies.

So maybe they have the Flash problem fixed in this build? Might be interesting to try it - just to see. In any case, it's relentless, but it's nice to see the progress. It's really quite impressive that it's overtaken Firefox so quickly.

I got another tweet from the Panic guys that they had updated Coda Notes 1.1 - and interestingly enough, the Safari Preferences panel handles all the updates for the extensions. It was a very pleasant surprise.

This afternoon I got a tweet that the great guys at Panic had released Unison 2.1 with a nice assortment of fixes - one of which was the little visual bug I noticed regarding the listing of the unread count at the bottom of the window. It's a nice app, and I wish I could spend more time reading the news, but it's nice to know that it's there when I have the time.

Today I spent a lot of time trying to come up with a really nice way to parse the exchange data. It's not as simple as it seems. I should say that it's not simple to make something that doesn't take hundreds of lines of code comprised of structs, if/then/else and switch statements.

Typically, you're going to get a message with a fixed header. In that header, the full "type" of the message will be encoded. Then you lay another struct on the message and it reveals the other fields you can pick off. With each of these messages, you could be looking at from four to ten different "patterns" and therefore structs. This makes for a lot of code to really parse out the data.

Couple that with the switch statements to know what struct to apply, and the code gets very large, very fast. One upside to this scheme is that the execution of this code is very fast. So... I wanted something that was just as fast, but was far more compact in the 'lines of code' category.

What I decided to try was a "tagging" scheme where I don't attempt to make complete sense of all the data in the exchange record, but simply indicate where each value is, and how to decode it. For example, if I create the struct:

  typedef struct {
    char       type;
    uint16_t   pos;
    uint16_t   len;
  } variable_tag_t;

where the fields are type, position and length, then I can create an array of tags that indicates where the values I need are located in the data stream:

  static variable_tag_t[]  tradeTags = {
    { 'L', 0, 10 },
    { 'S', 10, 15 },
    { 'D', 25, 8 }
  };

and I can read this: I have a long int at position 0 that's 10 characters long, a string starting at position 10 for 15 characters, and a double at position 25 for 8. It's not bad, and I can see a lot of good with this idea.

I can create constructors on the messages that take one of these arrays and the data from the exchange and use it to extract the ivars from the data. This way, the order and count of tags is fixed by the message's needs, but the location and size of each value is dictated by the specifics of the exchange.

It seems like a decent idea, but I'm going to have to make several more messages and look at a few more exchange message definitions to make sure that it's really going to work out. I certainly like the compactness of the scheme. Some will argue that it's all hard-coded numbers, and that's not good - but how much different is this than a bunch of structs? Either is a hard-coded definition of the data organization in the exchange data. This just happens to be more direct.

We'll have to see how things work out.

Today I spent quite a bit of time really fleshing out my variant class today. I needed to have a lot more functionality in the code as it was going to be an integral part of the ticker plant I'm working on. I needed to write a bunch of tests, and each new test uncovered either a compiler issue - like needing a new version of a method for a different use case, or a real bug in the code, which I had to fix.

Overall, it was a pretty good day, but it was all spent making the class a lot more useful to the developer that would be using it. Which, of course, is me.

This morning I saw that Acorn 2.3.2 was out with an impressive list of bug fixes and enhancements. This app has become the only real image editor I use, partly because my needs are pretty simple, and partly because this app is written for people that don't use Photoshop daily. It's easy to use, and that's the point. Excellent app.

Today I messed around with the boost::variant problem I'd been dealing with lately trying to get the code to compile and work properly. Finally, after about five hours, I gave up. It's simply too hard to get working, and even if I did, the maintenance costs of dealing with these kinds of compiler errors would be far too high for a junior developer.

So I took a very different track: I simply wrote my own. Honestly, it wasn't all that hard. I took out the definition of the boost::variant, and in it's place I put a simple union:

  private:
    tVariantType      mType;
    union {
      std::map<std::string, variant>    *mMapValue;
      std::list<variant>                *mListValue;
      std::string                       *mStringValue;
      int64_t                           mIntValue;
      double                            mDoubleValue;
      uuid_t                            *mUUIDValue;
      bool                              mBoolValue;
      error_t                           *mErrorValue;
    };

and then all the setters cleared out the old value and replaced it with the new. It's something I've written before, and so I knew a lot of the pitfalls to avoid. But it's not as nice as a stack-based template version. When I change values I'm hitting the heap for new space. While this isn't horrible for a lot of applications, it kills performance when you're trying to do something really fast.

Still... this is far easier to understand, and once we have it all buttoned-up, there's no real chance of a leak, and it's a solid way to handle the variant problem.

Once I got the main part written, I was able to attack the serialization and de-serialization schemes for this guy - based on the work of another group who has defined the scheme we'll be using. It's decently flexible, and should be really nice to use across the board.

Still lots of testing to do, but I'll get to that tomorrow.

This morning I saw that WordPress 3.0.1 was released, and SimpleScripts at HostMonster was telling me that it was time to upgrade. The upgrade took all of 5 min for all my journals, and when I was done, I was once again, safe for the time being. WordPress gets hacked a lot, and I try to stay up to date with it just because of that fact.

This morning I noticed that Google Chrome dev 6.0.472.14 was released with the exact same release notes as 6.0.472.11 - "UI tweaks and no Flash loading". OK, I can see it's not a really exciting time in the Chrome group, but I wonder why they are doing all these releases and not fixing the Flash loading. Probably won't ever know...