What's it all about, Alfie?

I noticed (while on vacation) that there was an emergency bug fix for WordPress to make 3.0.4, and it was quickly available on HostMonster, so I took the time to make sure that everything got upgraded nicely. It's not a lot of work, and there were plenty of ways to know that there was an update in the offing, so it's not bad, but it is sad that WordPress is such a target of goofballs that it takes eternal vigilance to make sure servers aren't abused. Sad, but true.

I'm to the point that I believe all my Broker re-write issues are solved. I can use no pooling on the sockets, and it works, and I can use pooling and it should work just fine. The guys working on the Java side of things aren't having as good a time of this as I am. Don't get me wrong, they're sharp guys, and I believe they understand the Java classes they are using, but the complexity of some Java libraries is really quite astounding.

Case in point: J2EE. It couldn't be harder to get something going on J2EE. Tomcat is hard enough - there should be something like JSPs or PHP that will allow you to place the java file in any directory and have things "just work", but I understand the "Java way", and while it's not perfect, Tomcat is awfully nice, and very capable - you just have to work within it's paradigm.

But this work we're doing on the sockets... Holy Cow! I"m glad I'm not doing the Java side. The C++ side has had it's problems, but still, it's very easy to see what's going on in the code, it's just a question of what's happening in the libraries. It's not a question of smarts, it's really a question of familiarity with the libraries. If you're new to the way it does things, as I was with boost asio in the beginning, it's a pain. But once you really know how to use the libraries, and how the library wants you to formulate a solution, then things get a lot better a lot faster.

I just feel for the guys doing the work on the Java side. At least one of them is new to this library, and this is a real trial by fire. The complexity is staggering to him, and it's clear to me that a really well designed C++ library can make a far simpler solution than a lot of the Java library/frameworks I've seen out there.

In the midst of working on the changes necessitated by the Broker's re-write, I found myself in a very nasty little problem. I am trying to do things quickly, and my test cases are often far worse than any real-world use is going to be, but they have served me well, and they were pointing out a problem I was having this morning.

If I created new updater instances for each request, and deleted them after they were no longer needed, I ended up with a very fast create-use-delete lifecycle. This lead to a segmentation fault in boost's io_service - specifically, in it's run() method. The core dumps were of little to no help whatsoever, and I was left trying to diagnose the problem from my end.

If I didn't delete them right away, but threw them into a pool, and still created new ones, only clearing out the pool at the end of the application, then everything was fine. It seemed like it was just the short lifecycle connections that was the problem. Very nasty.

The seg faults weren't on anything related to boost asio, either. They were on the line right after the context switch after the closing of the socket connection. I spent hours debugging the code to find that guy.

I came to the conclusion that there was something in the io_service that wasn't getting a "chance" to handle the socket closing before I deleted it. So I changed my code ever so slightly. Originally I had:

  if (si->second.pool.size() >= MAX_UPDATERS_IN_POOL) {
    cLog.debug("[recycle] the pool is full - deleting the updater");
    delete anUpdater;
  }

to:

  if (si->second.pool.size() >= MAX_UPDATERS_IN_POOL) {
    cLog.debug("[recycle] the pool is full - deleting the updater");
    anUpdater->mChannelID.clear();
    anUpdater->disconnect();
    // we need to let the io_service have a go at it
    boost::system::error_code  err;
    mIOService.poll_one(err);
    // ...and now we can successfully delete the updater
    delete anUpdater;
  }

The difference was stunning. No more crashes, and the code was rock solid every time I ran it. Amazing. I'm going to have to remember this. It's like a little context switch for the io_service so it can detect the close of the socket before it's deleted.

Several things to finish on the re-write, but it's getting close now. Nice.

As I was doing a little testing this morning I realized that the Broker wasn't working like it should. When I pinged a few folks, it was clear that there had been several changes I hadn't been told about and so I had a very abrupt detour to update all my Broker-facing code to make sure it was all working.

Some things weren't too bad - the codec's code for a map went from 'M' to 'm'... the Dates are now passed as microseconds since epoch... simple things. But then I quickly found that there were some that weren't simple at all. Not in the least.

The protocol for service registration changed. The interaction with the registration service itself changed. The recycling of open sockets wasn't working - in short, it was a massive change to the codebase.

I've spent the bulk of the day getting things converted. It's touched a lot of the code, but I think I'm getting very close. There are still a few things I need to hammer out, and I'm hoping to get to them in the morning. The prickly one is that the 'close channel' message - that which indicates that the socket is clear to be recycled, can, if the conditions are right, close the actual socket. When that occurs, it's happening too fast, and the object is getting dumped before it's work is done.

I need to figure out how to fix this so the events happen in the right order.

This morning I'm continuing to work on the functions for my lisp-like parser. What I'm working on now is really the functions themselves and not how they fit into the other classes of the parser to compute values. It's a little slow going at first because I want to duplicate as much functionality as possible from the Java-version, and as you'd expect, there are absolutely no comments in the Java code. So I'm ending up chatting to one of the developers on the project and he's giving me the details on the functions.

Nothing hard, but it's slow going. I'm hoping once I get a few done, it'll go a lot faster.

Heh... that's optimism for you...

I've been in a lot of meetings all afternoon, but in the end, the good outweighed the bad. These other groups had their own biases, of course, and no one wanted to "just give in" and use the stuff I've been working on, but that's really understandable. After all, a few lifetimes ago, it wasn't until after my ticker plant was proven for the most demanding client (my own risk engine) that other people started to see the value in it. SO it goes... I'm going to have to prove it, and I understand that.

Still... they had several good points:

• If we conflate the messages by type and instrument, how will they know that any conflation happened? It seems reasonable that they should be given some kind of information about the conflation being done.
• They'd like to be able to know what "stream" the message came from. There's 24 OPRA channels, and they'd like to know which one sourced this message. That's fair, but I have no intention of putting that in the message - that's going to be a client-side look-up method to keep the messages small and useful.
• The exchange is an important part of the conflation - meaning they want to have one quote message for the AAPL Jul 350 C per exchange. Yikes! I hadn't planned for that. That's going to make the cache much larger, but so it goes.

Many of these aren't too bad to add, and it's reasonable to get them out quickly, but some of the others are a little trickier, and it might be better to think of the real solution to the problem as a completely different kind of answer. For example, what about having a message with all the exchange prices and sizes for a given instrument. Then, when it's shipped out, the user gets all the prices - at once. It's got possibilities.

It's late and I need to catch a train, but it's a lot to do in the coming weeks.

This morning I've been working more on the lisp-like parser for embedding in my projects, and it's coming along nicely. I realized that I needed to make my collections use pointers if I wanted to really use polymorphism, which just makes the housekeeping a little more, but other than that, it's fine. With that, I'm getting the right messages dispatched to the right instances, and things are moving right along.

I have been adding the function class to the code this morning, trying to get the evaluation of functions going. It's just a lot of little details and lots of writing. The base classes aren't bad, but every time you start another, you're looking at a good bunch of typing. I just need to get the first version hammered out and then I can look and see if I need to make it faster.

The rest of the day is in meetings, but it's nice to get it all moving forward this morning.

One of the many potential next projects for me might be a greeks server - like I've worked on in the past. The difference here is that it's targeted to me a lot more flexible than the old MarketMash system - this one is going to allow for the clients to give it "mutations" to the center state so their numbers can be as good, or bad, as they want. I've fiddled with a lot of different schemes for specifying this kind of mutation - including muParser. But those don't seem right in this context.

The choice for this seems to be a lisp-like language that is in use in some other projects in The Shop, and it would be a natural fit to have a C++ version of this for any C++-based projects. Right now, the only version is a Java-based one, and while that's nice, it's also heavily customized to the application it sits in. What I'm looking for is a simple parser that can have variables, constants, functions all added to it - augmenting a nice set of default capabilities so the user can make of it what they want.

The language isn't really hard. The expressions are really pretty simple:

  (set boo 5)
  (+ boo 10)
  (* 12 5 3)
  (avg (min boo 5) (max boo 5))

stuff like that. Each expression has a function as the first argument, and then a series of items - which can be values or other expressions. I've just decided to make the Item a base class, subclass Value and Expression, and then start the coding.

Today was all the parsing of the input text, and it's pretty nicely compact. I'm pleased with the way it's looking. Tomorrow is the functions, etc.

Seems Google Chrome dev 10.0.612.3, out today, fixes a regression test with the RegEx in the engine, but that last part is an assumption that wasn't explicitly stated in the release notes. In any case, it seems that we have two regression test related fixes in just a few weeks. The engineers at Google are slipping... maybe they've had to lower their standards to get employees?

OK... maybe just sloppy - but which one is the worse explanation, really?

Today was heavy with meetings, but I was able to get some nice work done on my ticker plant - the Java guys were interested in using my security name to 128-bit int converter as it was non-lossy and bi-directional. There are times they want to use a number but not loose the ability to pull out different aspects of the instrument. What I went with was a subclass of my UUID where I added in a lot of nice helper methods:

  tInstType getType() const;
  bool isStock() const;
  bool isOption() const;
 
  std::string getUnderlying() const;
  uint16_t getExpiration() const;
  float getStrike() const;
  bool isPut() const;
  bool isCall() const;

where I've dropped the comments for the obvious nature of these methods. The point is that we aren't going to decode the entire 128-bit stream to get the one bit that's the Put/Call flag, so these guys allow you to get the one piece of data you want without having to convert it all into a string and then parse that.

I got this all in, and the tests were just fine. I even had the operator=(std::string) written so you could do something like:

  msg::ng::secID_t   id = "O:GOOG:110214:74550:C";

and it'd parse the string into the 128-bit number and be ready to go. Should save a lot of time.

Finally, I added this as a possible component of the variant class so that it could be put int he codec for the messages to the Broker. This will really help move things around as we'll be able to handle getting this as a data type in anything coming from, or sending to, the Broker. Very nice.

But then there were more meetings, and some were really draining. Still... it was nice to have a little bit of work under my belt for the day. Made it feel like it wasn't totally spent in meetings.