Rewrote the NBBO Engine – Better, Faster, More General
Today I took on the task of re-writing the national best bid/offer (NBBO) engine in my codebase. This is significant because this was very fast, lockless (for stocks), and was a critical part of the data feeds. It's just that important to get the data right. But it had a limitation that was a killer - it was based on the idea that each instrument belonged to a "family" - rooted in a stock or index. The problem wasn't bad for stocks, and the options weren't too bad, but we are starting to get instruments like spreads that aren't based on one family, and therein lies the problem.
I solved part of the problem by making these spreads have a 128-bit ID value like all the other instruments, so that they might fit in a nice 16-way trie. I just then needed to fix the NBBO engine to use this trie as opposed to the family-based (name) trie.
There were a lot of little details to pay attention to, but for the most part, it was a smooth transition. The data is now completely lockless if we want, but for now there's a spinlock at the instrument-level to make sure that we don't update the NBBO data improperly. However, this is likely never to happen as a single symbol comes from one feed, and one feed alone, so it's a single thread. However, it's possible that someone could make a multi-thread, single-feed, and in that case, it's possible to have two threads with the same instrument. So I'll leave it in for now. Just to be safe.
The upshot of this change is that we're now ready to handle any instrument that fits into the 128-bit ID scheme. Also, because we got rid of the map for options within a family, it's faster for options than before. Sweet.
Plenty of testing to do, but it's a great start.