What's it all about, Alfie?

Today I did a lot of work helping people get focused on what we should be doing. For my part, we needed a Time and Sales Service where each option trade sent to us by the exchanges needs to be captured, a few implied volatilities calculated, and then shot out to the waiting clients. This is really orthogonal to the rest of the greek engine, so I wanted to tackle that so it doesn't slow down other people, or divert them from the primary goals.

The design is going to be very simple: a service that takes an Option and a Print message and queues it up to be calculated and sent out. Very simple. The design is pretty simple: have a struct (object) that will hold all I need from the two incoming arguments, fill it on entry, queue that, and return. All this should be very fast as it's all just ivar calls to the base objects.

Then the queue has a processing thread, and will take these off one by one, calculate the values, and then generate the messages in the legacy and new formats so that we can feed both the old and new systems. Pretty simple.

This morning I had an interesting talk with my manager about the greek engine and what we might want to focus on in this first release. Rather than think about the general case of handling the entire market, which certainly needs to be done, let's focus first in the idea that the largest percentage of clients of the engine will be looking at a single family, and wanting to manipulate that one family to see different what if scenarios. In the end, we'll need to make an uber environment with all the instruments ticking, but that's a limit case, and will be able to be extended from this initial one-family case.

In addition to the what if work, the engine needs to handle the explicit requesting of options so that the client can say "show me everything between the Dec 150 and Dec 250 expirations". This "windowing" is done all the time to minimize the data traffic, and building it into the server means that it doesn't have to be built into all the clients. They just have to be able to pass the request to me, and then update the window with me as the scrollable region changes. Pretty neat.

In the end, I think this is a much more useful direction to go, as this is going to really change how we move data around the organization. No longer will clients have to take "the market" to get a few things. Now, they'll be able to ask for just what they want. Nice.

This morning I'm really having a hard time with a lot of things. Questioning a lot about who I am, how I approach things, how I react to things and people... and it's not at all fun. Not in the least. I'm wondering if my passion about things is really nothing more than my little rationalization for my actions. Does anyone really care?

Which reminds me of the great line in 1776 when Abigail is talking to John Addams and says:

Have you forgotten what you used to say to me, I haven't. Commitment, Abby, commitment. There are only two creatures of value on the face of this earth - those with a commitment and those who require the commitment of others. Do you remember John?

When I think of that I realize he's right, and so am I. OK... maybe neither of us are right, but I'd rather be wrong - like that, than sit around thinking there's nothing that can be done, or nothing that matters. Life matters, if only to me.

I don't expect a lot of people to agree with me, but then again, as Cher said, This life isn't practice for anything.

Today among the many little things I had to do for my greek engine project, I added the ideas that for each tradable instrument, there really was a traded volume per exchange, and for the Option, there is a quote per exchange. These are going to come in very handy when we have to look at the volume sliced and diced across different exchange sets, and when the consumers of this data need to see the latest quotes for an option across every exchange.

I was pretty happy that these both went into the design very well, and I spent probably a little too much time adding nice features to the basic data, but I think in the end, it'll be a better way to present the data, and it'll certainly make a solid example for the rest of the team as to how to do it well.

Besides this, it was a lot of helping others along with problems, and I realize that it's something I should be doing - even though it's not my favorite thing to do.

I was in a meeting today with a parter at The Shop and was really stunned at his rudeness. This is a guy I've been in meetings with before - with other people, and he wasn't at all like this. Sure, the other people were other partners, but I had no idea that this guy's sense of professionalism was strictly dictated by the people in the room. Pretty amazing to me.

It says a lot about a person when they act entirely different around people who they see as "below" them as opposed to those that are their equals, or superiors. And I'm not being exceptionally picky here - this guy was arguing with me about why I should be listening to anyone other than him. I merely said that since there was a difference of opinion, that I thought he and this other person should get together and work out a consensus decision. He seemed offended at the idea.

He was totally uninterested in listening to my ideas - getting up and walking out on two occasions. I mean Really? This is how an adult acts? Holy Cow! I expect better from my 12 yr old daughter. This guy is supposed to be leading this company. I wouldn't follow him to Dairy Queen now. This guy has done nothing other than to breed contempt for him in my eyes.

Who knows... maybe they just killed his dog, but I'm pretty sure this guy is just an idiot and got to this position by spending so many hours here the people think he deserves the role "just because".

Who knows? Maybe he's really quite smart. But his immaturity today spoke so loudly it's hard to imagine that his brilliance would be heard by anyone in that room.

Today I got around to writing the iterators on my thread-safe Instrument list - hadn't done that before, and it was actually fun. Geeky, yes. But fun nonetheless. I just needed to get the style of the header right, and a first cut at the basic iterator and they were coming pretty quickly.

I wrote a simple test program as well to make sure that they were working, and it was pretty impressive (to me) that they worked right away. No problems. Sure, they weren't all that hard, but still - for not having done any in the past, it was a treat to get them done so quickly and done well.

There's a ton more testing that these guys need in order to be sure they are going to perform under load, but I can pass that off to a junior developer that's got a decent eye for details and he'll expand the test app to be a testing monster.

Sweet!

This afternoon I started work on the next component of the greek engine project - a list of pointers (instruments) that is thread-safe, and has iterators that are never invalidated. It's the combination there that's really tricky... how to make a list that has many iterators on it - none of which will be invalidated on an isert or removal operation on the list.

Yeah, that took some thinking.

My initial idea is to have a fixed-length list of nodes. Each node has a value, a 'valid' flag, and a spinlock. In order to add to the list, you need to scan the list looking for the first node that is not valid, but who's try_lock() is successful. You can then place the value into the table. Pretty simple, really. The locks don't effect anyone else, and things are pretty smooth.

The iterators are going to be something where I can step through all the valid values and optionally lock on that node to make sure it gets "stepped over" by the other iterators. It should make for just what I'm looking for.

Lots of coding to do, but I think it's all planned out and should work. I just need to actually do it.

This morning I wrote the skeletal InstrumentFactory and InstrumentMap for the Greek Engine to get some of the other guys in the team helping out. These two classes work together to construct and then hold the universe of Instruments that we'll be working with. It's a model I've used in the past, and it really seems to work pretty well. I just need to get working and figure out how to make a thread-safe, lockless, list with iterators so I can run through the stocks in this map for calculation purposes.

No rest for the weary...

Today I spent my own coding time building a client for the Position Service at The Shop. It's a TCP-based service that you can subscribe to and receive updates from. It actually looks to be pretty darn sane - which is a little unusual for the state of things here, but I'm willing to roll with it. The wonderful fact is that I've been working with boost asio enough now to be able to ship up a nice TCP-based client in just a few hours. It's pretty powerful stuff.

Throw in a little thread and memory block pooling and I have a client in very short order. The only problem I had was that a critical part of the functionality could only be tested during trading hours, and I didn't get enough of the code done to test that by 3:00, so I'm going to have to test that tomorrow and see what needs to be finished.

In all, though, a pretty decent day.

Today I needed to tackle the 2-D grid and interpolation in as efficient and re-usable way as possible. In previous attempts, I've used special data structures to hold the data points because the interpolation was handled differently. This time, I wanted to take advantage of templates for the z-value so I could re-use the class for doubles as well as entire packages of greeks. In order to do this, I decided that it'd be better to use boost::unordered_map and have the key be the coordinate value, and the value be the function value. When I needed to handle two dimensions, the outer map values were really just axis/function values, and with a simple spinlock to protect the grid, we were done.

All this coded up pretty easily. We are going to stick with a four-point interpolation scheme so the code had to "locate" the four nearest grid points to any given point, and then allow the caller to ask for the interpolated value anywhere on the grid. It's just a few detail functions, and it worked out pretty nicely. The final point was having a templated function that took the (x,y) points and the z values, and then returned the interpolated value. This would allow me to make one of these for double values, and another for the greeks package.

Because it was completely isolated from the rest of the grid, I could change the interpolation from a simple linear to a Taylor Series at any time, and that's exactly what I have planned for the greek payload. After all, I know what the x- and y-axes are in this space, and I know that within the payload I have the derivatives, several powers worth, and that will make it a lot easier to space these points further apart and get reliable interpolated values.

All in all, I have to say I've enjoyed coding this up. Not everything today has been this fun, but at least this was a good note to end the week on.