What's it all about, Alfie?

Today I started, then stopped. Then restarted work on a National Best Bid/Offer engine for my ticker plant system. Basically, the rules are pretty simple: you keep track of the ticks for an instrument by exchange - keeping the latest bid/ask data for each exchange. What you might end up with after a few ticks is something like this:

The rules say that you look at all the bid prices - pick the largest one, and then sum all the bid sizes for those exchanges where the price is the "best bid". In this case, the best bid is 12.50 and the sizes are (25 + 20) = 45. For the best ask (offer), you look at the minimum asking price, and sum up the sizes that match that value. In this case, the price is 13.75 and the size is a simple 20.

So, there are a lot of ways to accomplish this. One is just to have a cache of all the ticks - organized by instrument, by exchange, and then when a new one comes in you toss out the old, recompute the new NBBO, and then if something has changed from the last time you calculated the NBBO, you send out a new message to the downstream clients. Pretty simple.

The problem with this is that while it's easy, it's a guaranteed two-pass system because you need to first find the best price (of either kind), and then sum all those sizes with that price. Not ideal. Also, there's the possibility that a Quote message from one of the exchange services will have a bid from one exchange and an offer from another. So it's not rally easy to keep the entire message - and I don't really need it. I only really need the data.

So what if I just kept the data?

  struct ExchangeData {
    // bids and asks are separate and organized by exchange
    float      bidPrice[26];
    uint16_t   bidSize[26];
    float      askPrice[26];
    uint16_t   askSize[26];
    // ...and this is the NBBO that's generated from it
    float      nbboBidPrice;
    uint16_t   nbboBidSize;
    float      nbboAskPrice;
    uint16_t   nbboAskSize;
    // ...and a lock to make sure it's all changed atomically
    boost::detail::spinlock   dataMutex;
  };
  typedef struct ExchangeData exch_data_t;

If I have this structure - one for each instrument, in some simple data structure (like, say, a trie), then I can forego the message retention, and just retain the data from the message and update the data in the structure.

But wait, there's more.

If I'm careful about what I do, I can update the NBBO values without having to make a two-pass update. What I can realize is that there already exists, for each tick, an NBBO, and if I just update that by subtracting out the old data, and adding in the new, then the values will always stay up to date. No need to do the two-pass calculation.

The code for just the bid looks something like this:

  if ((bidExch != '\0') && (bidPrice > 0.0) && (bidSize > 0)) {
    // get the exchange as an index into the arrays
    uint8_t    idx = bidExch - 'A';
    // see if we need to remove the old bid from the NBBO
    if ((myExchData->bidPrice[idx] == myExchData->nbboBidPrice) {
      myExchData->nbboBidSize -= myExchData->bidSize[idx];
      newNBBO = (myExchData->bidSize[idx] > 0);
    }
    // see if we have a *new* NBBO bid, or maybe just an update
    if (bidPrice > myExchaData->nbboBidPrice) {
      myExchData->nbboBidPrice = bidPrice;
      myExchData->nbboBidSize = bidSize;
      newNBBO = true;
    } else if (bidPrice == myExchaData->nbboBidPrice) {
      myExchData->nbboBidSize += bidSize;
      newNBBO = true;
    }
  }

and with this code, I should be able to maintain a decent NBBO engine with very little latency through the system. I still need to write a few things and test it all out, but it's a really promising start, and I'm excited about the possibilities.

Sweet idea.

UPDATE: This logic isn't going to work. What if I fall out of the NBBO? This code will remove the contribution, but if I'm the only one, then we're left with a bad price and a zero size. I need to fix this. But I still think it's salvageable. The only case when we will have to scan is when there's only one exchange in the BBO, and that one is out. Then, we have to find the next best, and that's likely a scan.

It was Halloween a few days ago, and that means two things: Pumpkin Pie and Tootsie Pops. I had both, and was lucky enough to have a slight surplus of the latter to bring a few to work. They are so fun. And tasty.

But Mr. Owl was wrong... it takes a lot more licks to get to the Tootsie Roll center of a Tootsie Pop. A lot more than three.

Today I was working on the Enrichment Server part of the Ticker Plant system and was making pretty good headway when the Chief Architect came up to me and asked me what it would take to get a complete, full-up, system going. "Well... I'm pretty close... I'd say a few days." I thought. "Good! Let's try for the end of the week." Sounds OK to me.

There were a few things I needed to get back into the source code for the tests. We'd done a significant change to the messages - ripping out the old 64-bit security ID in favor of the std::string security key, and the group that was going to use this needed to have the security ID for trading reasons. So I had to add that back in, but only in the Client code as it makes no sense to slow down the upstream systems for the few that need this feature.

So I started digging into my git repo to find the check-in where I took it out. I have to say, for a "universal" client, gitk is not bad at all. It's pretty decent for being platform neutral. I was easily able to find the check-in and then look at the diffs and see what I needed to add back in.

It took a little bit, but it was not really hard. Just a little time.

Unfortunately, the Broker and it's infrastructure is not up, so I can't test. But when I can, it'll be nice to make sure all this works. Then I just need to find memory for my box as it's got far too little to be effective for a complete SIAC feed... but that's a task for the management types who control the money.