What's it all about, Alfie?

For the last few weeks I've been slugging through a lot of code written by guys that are a little junior, and end up making junior-level mistakes. These aren't horrible problems, but they certainly have set me back a bit as far as getting the project done. Each time I run into a style of problem, I try to point it out to the guys to say why this is a problem, and why not to write code this way in the future. I have a feeling this has mixed reception, but I'm trying to help them become better at this craft, but it's often a painful process - for them and me.

Today was some really brittle code. Make one little change to one method, and I have to change another. This is often the sign of bad interfaces. If an object is well-defined and it's methods are well thought-out, then you usually add methods to a class, or make different classes. But when it's sort-of thrown-together, you have to change the calling parameters for some methods, and pass in complete objects, to get the added behavior.

This is brittle code. It's incapable of being stretched, expanded, changed, without major changes to the surrounding code.

It's hard to work with this, and while I'll end up re-writing it all in the end, for now, I need to work with it - if I can, and get something working in the short-run. But it's all a bunch of mental notes for what to come back and clean out after the initial release is over.

Today I've been working on putting a cache back into all the price feeds for my greek engine - and then persisting them to the config service. This is important because when I reload the instruments, it'd be nice to be able to hit a cache and have the instruments "priced" right away. I tried hitting the external ticker plants, but that was nowhere near fast enough. I needed something that was a lot faster.

Then again, I also realized that the first deployment of the engine will not need the external ticker plants, so it made sense to spend a little on the memory footprint, and put the caches - with persistence, into the engine. It's going to pay off.

Again, the speed was pretty nice, but the sheer volume of data is pretty big, so it's going to take a few seconds to persist all the data, but that's OK. It's going to be done in an off-loaded thread, so it's not that bad.

Getting closer. I now have good prices, and persistence that makes testing fast... it's really starting to come together.

This morning I finished up my reformulation of the persistence of the serialized instruments to the configuration service (backed by a mongoDB). The results this time were really quite nice. I was able to serialize everything in about 1.5sec. Very acceptable. Deserialization on the app start-up was fast as well, and so today I'll be able to get a lot more testing done simply because I'm able to get a lot more tests done in the same amount of time.

It's going to really pay off. Nice.

This morning I noticed that Google Chrome dev 13.0.772.0 was out and the release notes show some nice progress. The latest V8 engine (3.3.8.1) as well as print preview are both nice. It's great to see them making progress.

Today it was all about serialization. The problem I've been having is that my tests hit the database for some 400,000+ instruments, and because of that, each test takes about 5 min to run. The database is not fast, but it's the fact that it might only take a minute to get the data, it's the fits and starts with iODBC that's hurtful, and the database is really in bad shape. So I decided this morning that if I could get serialization of my objects going, then I could use persistence, and that would make my tests a lot faster.

So today it's all about serialization.

It's not all that hard - just a lot of details and making sure things work properly. The big problem is that when working with instruments, there's a lot of references going on, and getting all that right in the serialization is non-trivial. Still, having seen it work for me time and again, it's definitely worth it.

At the end of the day, I had something that worked, but it took far too long to persist everything. I was serializing each instrument family (a stock and it's options) independently, and saving that to it's own mongoDB document. This was just too slow, as it was taking 30 sec to write out all the data. In the morning I'm going to pack things a little differently and put as much in a document as I can. This way, it's fewer, bigger documents, and not so many small ones.

Today has been just a lot of code re-writing. It's all about not being hard enough on the more junior members of the team. I want to lead them down the path of doing the right thing, but along the way, it's painful when they keep missing the points I'm trying to make.

Documentation is easy to get right - it's what you're trying to do. Coding it up is a lot harder. But why then, is it that someone who isn't doing the latter, fails to do the former? I'm trying to teach them that rather than making two mistakes, get the documentation right and then you're only going to make one - max. I'm afraid it's just going to be a long and slow process.

But I'll get there, it's only a question of time.

I've been on a very accelerated timeline for a project that has the potential to make a very big impact to The Shop. But I've been amazed at the number of times that I've wanted to scream at the guys on my team about making sure they're doing the right things. I guess I take it for granted that developers know what it is they should be doing. In general, I've seen that not a lot of folks really do get it, and so it's clear why so many people fall back on process.

It's the way to ensure that things get done. But it's based on the idea that you don't really have skilled labor - you have a bunch of advanced beginners, and they need all the rules and regulations because that's how they code.

I understand why people push for this - it makes it appear that you can get good work out of marginally skilled workers. But you can't. Not really. The rules can't cover every possibility, and there will be things that fall in the cracks. There's no silver bullet - you have to have good, skilled people, and then you don't really need the rules.

Unfortunately, I'm not "there" yet. So I have to try to make the most with the people I've got.

Today has been a hoot. I've gotten to the point that I'm able to glue in a major piece of the puzzle into the framework I've got: I can now load up the slice of the universe I need to cover and the clients can hit it and get what they need while at the same time not crashing the system with nasty allocation/deallocation errors. OK... there was a little bug hunting I had to do for that last part, but still - I got it.

The only thing I'm missing now is the ability to do the calculations and generate the messages. I'm letting another guy finish that and hopefully, he'll be done very soon and tomorrow I'll be able to have a complete functioning system. That would be great.

Great progress.

This morning I saw that Google Chrome dev 13.0.767.1 was out with a few nice updates, and something I hadn't noticed in the past - significantly better restart rendering performance. I mean Wow! When I restarted 13.0.767.1 the first page was there. This is in start contrast to taking a few seconds to get that first visible page up. Now I don't know if it's just cached, and that's OK, but either way, the impression is that it's UP, and fast. Very nice UI choice.

Today I spent all day re-writing an instrument loader because over the course of the last two weeks the junior guy on the team that I handed this project to has made a complete and total mess of it. Each day, I asked him how things were going. Each day I got the same "looking good... almost done". Each day, I fear, it got a little worse. Finally, the project was waiting on him, and I had to step in.

I should have done so much earlier. My fault.

I like this guy, I really do, but this was a train wreck, and he should have asked for help after the first few days. There was shared state in the loader methods that would have failed had even two clients tried to load anything. In the tests he wrote and used, this never came up, but it took me less than 5 minutes to see this problem, and realize the depth of the problem.

This is the second guy on the team that I needed to put on test writing duty for a while. I should not have allowed it to get this bad, but I did, and that's my fault. I don't believe I shoulder all the blame, as I asked how it was going and he didn't speak up - but that's inexperience for you: overestimating their own skills, and underestimating the problems they face.

So today I was coding like a madman trying to get something that worked for the problem at hand. Thankfully, I got it working, but it took a full 13 hour day to do it. From here on out, I'm going to keep his work to finishing the skeletons that I write for him. That way, I can limit the damage.