What's it all about, Alfie?

It's now 5 days before I'm supposed to be delivering a working greek engine to QA at The Shop, and while there's still a slim possibility that it'll work, I have so many doubts about things that aren't done it's hard to really think it's likely. Then again, getting even close is a victory in my book - after all, I was only given 4 weeks to do this, and I would have bet it would have taken twice that at a minimum.

Still... I've got five days, so let's make the most of them...

Today has been another long day in fleshing out the implementations of the headers a co-worker has put together for the greek engine. He's had the most experience with the model in use, and the gotchas with the data going into said model, so it makes sense for him to define the headers to a decent point, and then several people can take it from there.

It's not particularly exciting work, but it needs to get done and seeing as how we're a week away from "being done" (so the target says), there's no time to get attitudes about what kind of jobs there are to do - you just do everything you can to move the project forward.

This morning I've come across a real possible good use for a C++ IDE - renaming a namespace in all the places it exists. I realized that when I had the namespace AppKit in another project and wanted to use it in the greek engine project, I couldn't have eng::AppKit because the compiler couldn't tell what to do with:

  AppKit::Message msg;

Is that AppKit on it's own, or in the context of the eng namespace? I knew which one it was, but there wasn't any way I could fine to make the compiler understand that fact.

Also, I knew that the namespace was badly named. I just didn't understand how widely these things would be used. My fault. So I tore into the code with Vim. There's lots to like there, but doing a massive change like this is just not all that easy. It's not hard, it just takes time.

If an IDE could do that, I might have reached for it. But I didn't know one, so I toughed it out.

This afternoon I got the combined pricing feed for the greek engine working. It's not that hard - just a combination of an existing UDP exchange feed and a receiver of the legacy data with a little NBBO engine thrown in to generate the NBBO quotes for these options. It's nothing that amazing, but it needed to be put together as a unit so that it's a lot easier to use than glueing all those pieces together in the greek engine.

Not glamorous, but it's got to be done.

Another late night trying to get what Joseph calls my Impossible Project done on time. I've finally been able to let the Broker sit for a while, and I'm on to getting the database abstraction layer for the static data we'll need in the greek engine working. I've got a good start on it, and SQLAPI++ really helps, but I wasn't able to really get it nailed down before I had to run for the train.

It's getting a little old - these sprints for the train. But hey... it's part of the job these days. I know it's a nearly impossible timeline, but it's what they've asked for, and I'll do my best to deliver it to them as they asked. That's the deal we all make in a business.

I've got about another hour left on this guy and then my first database table will be done and I can hand it off to someone else for them to work in all the other tables that need to be included in the interface. Once I have all this stabilized, I'll work with the static data vendor group and see about converting these into more generally usable Broker-based data services.

Just for right now, it's enough to get them loaded from the database. We can get the rest done later.

I've been working on the conversion of my Broker code to the new replica set-compatible erlmongo driver. It's not horrible, but the worst part is what the worst part always was - getting the data in and out figured out so I can write the correct matches in the code.

They store the arrays differently, and they don't like the long binary-data strings that I've been using, but after I figured out all the gotchas, it's working again. It's just not easy.

Plus... this is more than half a day shot on something that I didn't need to have done had they not switched the database clustering... and I'm on a week-and-a-half deadline. Not ideal at all.

But it's done.

This morning my newly re-written Broker was not working. Hmmm... so I look at the logs, and it's not too hard to see that there are timeouts on the emongo driver. This was the driver that I used to get from erlang to (and from) the mongo database. Now all of a sudden, it's giving me timeouts. I had no idea.

When the guys that handle the mongo DB arrived today they said that they reconfigured the mongo DB I was using from a single server to a replica set. This is a fault-tolerant mongo DB setup, and in general, I'm a fan of things like this.

But that's not how today went.

Oh no... turns out the API into a mongo replica set is not the same as a single server, and moreover, the emongo driver doesn't support the replica sets. Fantastic!

So now all the work I just finished has to be re-examined in light of a new erlang-to-mongo driver: erlmongo. This guy is capable of dealing with the single server and replica sets modes of mongo DB. This is good, but the API is very different, and most importantly to me, there are no connection pools. If you want to talk to a database, you have to open a connection to the database and then hold onto it.

Given the realities of erlang and persistent state, this isn't really a possibility. We need to have a module that's a connection pooler, and then be able to set that guy up and let him deal with opening the connections, etc.

So here I am, fixing code I just wrote because some other group decided to change the database into an incompatible format. Lovely. It's unavoidable, as it's their group, and their database, but it's a real pain - right on the heels of just getting it done.

This morning I saw that Google Chrome dev 12.0.725.0 was out and the release notes say it all:

All

• Updated V8 - 3.2.6.0
• Spring cleaning in the code, lots of code cleanup and refactoring under the covers

It's nice to see the clean-out of old code. I'm sure there's a lot that's sitting there - like maybe the H.264 decoder... OK, enough of that. In any case, at least they got the icon back. Baby steps.

I finally finished the erlang services for The Broker. I was able to get my ticker plants hitting it and all was running. It's been a lot of work, and not a lot of "new stuff" - face it, it's a re-write, but in the end, we have something that's a lot better than the old, and it's going to be paying benefits long into the future.

Good work.

Today I made some real progress on The Broker. It's still the login service and the configuration service, but I'm getting things together. It's all skeletoned out, so I know what I need to do, and what I need to make it work - in a minimal sense.

Things are looking a lot better today.