Merging Live and Historical Message Streams
Today I've been working on the problem of merging the current tick data stream with the last known ticks for a given message type and instrument. The reason for the merge is that when a user of my ticker plant client (TPClient) subscribes to a certain set of messages, we need to be able to provide him with the last known versions of those messages as well. If the market is closed, this represents his only data. If the market is ticking, then chances are he's going to get a more recent message, which is why we have to properly filter them out.
I've got the filtering working out because it's pretty easy - look at the type of the message, and then have a simple std::map of conflation key to timestamp. If a message comes in, and we have filtering turned 'on', then check to see the last timestamp we saw for a message like this. If we have a newer one, send it. If not, don't. If there's no filtering, then send it regardless.
This allows the client to decide if they want to see the messages, and filter them on their end, or allow the TPClient to handle it for them. It's all pretty simple.
What's proving to be a touch more difficult is the providing of the messages for filtering. It makes most sense to have the messages pulled from the cache when a subscription is done. Also, it makes sense to have the criteria be the same. It's also been established that this request will be a 'call' to an MMD Service that will be the QuickCache within the ticker plant's exchange feeder. What's not so clear to me now is the way I'm going to be able to map the request to the MMD service, and how I'm going to get that data back into the TPClient's filter's onMessage() method. But I know I have to do just that.
I'll tackle that tomorrow.