Wonderful Solution to a Locking Problem – Merging the Streams
The past couple of days have been about speeding up the processing of the exchange data through my feed system. Because the code for the decoding of the messages is fixed, specifically for the big OPRA feeds (using OPRA's FAST decoder), most of this is accomplished by re-organizing the data flow and data structures. One of the things I had done a while back was to have the two channels of an exchange feed put their datagrams in separate queues, and then have one thread empty both so as to remove the need for locking on the sequence number arbitration code.
The problem was that this required twice the time to process the data through the decoder because both A and B sides went through the same thread. This can get to be really quite nasty. For example, on an OPRA feed, it takes about 40 usec to decode an average datagram, and at 50,000 datagrams/sec we're looking at 2 sec to process one side, but this design would have to do double that work. Nasty. Lots of buffering.
The solution is to have one thread per datagram stream. That immediately cuts the processing time in half. The problem is that we then need to lock for the sequence number arbitration. Nasty. Then I had a flash - merge the data!
First, tag one of the channels as primary, and have it control the arbitration. Every other channel decodes it's datagrams but then instead of trying to have the thread send it out, have that thread put the decoded messages into a queue that the primary will process as soon as it's done with it's datagram. The arbitration is very fast because it's as simple as checking the sequence number and a few flags. It's the decoding that takes the time. With one of the FIFO queues, we can have multiple non-primary channels, and have the primary take the results off and send them out.
Even more importantly, the primary can be the primary feed line of the exchange, and that makes things even better as the secondary feed is really only needed when there's a failure of the primary. What we've done then, is to make it more like the "normal" feed with a "backup" just in case.
Very neat.