Today I spent most of the day trying to flesh out the infrastructure for a few concrete message classes of exchange data in my new ticker plant. The initial work I'd been doing was focused on simple message types, and just about anything would work there - so I used my simple 'Hello' and 'GoodBye' TCP handshaking messages, and they worked fine. But now, I've got people wanting to test components that require the sequence number from exchange messages, and that means I needed a lot more infrastructure than I had built up.
The first thing was the idea of the conflation, or 'compression', key on the messages. This is something that I know I'm going to need in the client code as the current system has historically had a serious problem of not being able to handle a slow consumer very well at all. I settled on a simple uint64_t as the conflation key type and added a very nice little string hash method to get decently diverse integer values from strings:
static const size_t __initialFNV = 2166136261U;
static const size_t __fnvMultiple = 16777619;
size_t Message::hash( const std::string & aString ) const
{
size_t hash = __initialFNV;
size_t len = aString.length();
const char *p = aString.data();
for (size_t i = 0; i < len; ++i) {
hash = hash ^ (*(p++));
hash *= __fnvMultiple;
}
return hash;
}
and from what I've read, this Fowler, Noll, Vo algorithm is pretty decent at creating a diverse mapping of strings into the integer space. It's not something I'm going to count on as unique, but it's decent enough for some message types.
Now that I had something to put into the getConflationKey() for my existing messages, I started with the exchange messages. I needed to make a base message for exchange data, and then a faux price message that I could use for testing. This is going to be something that is very close to a "real" message, but will be strictly used for testing.
Why? Because I can control everything about it and it'll never change. This is the kind of test framework we need for the performance testing and comparison runs. True, we'll also need real-world exchange data for some additional testing, but that doesn't negate the value of this kind of test framework.
Then I realized that I needed to make a final determination on how to integrate the "parsing" of the external data into msg::Message instances. I decided to go with a very simplistic API. The MessageFactory has three basic methods - the create() method to take external data into our messages, the extractSequenceNumber() to extract just the sequence number from said external data, and the masquerade() method to be the inverse of the create() method and attempt to make external data from the message.
I then had to put this all in place and write the faux price message and it's data conversion class. It was a lot of coding and I didn't quite get it all done today, but should be able to finish it up tomorrow.
