What's it all about, Alfie?

Well... today was the first day of the real re-tooling of the server for 24hr operation. Up till now, it's been restarted each evening, but with the needs of the Hong Kong group coming so close on the heels of the end of the Chicago trading day, it was time to look at a more continuous Server. The first thing I needed to do was to extend the protocol between the server and it's clients to allow for something other than the 'instrument update' notification. I had to add in the capability for the server to send a 'remove instrument' notification when a new underlying is reloaded and the list of derivatives do not match those that used to be on the underlying, and so there's an instrument that has been removed from the server and needs to be removed from the clients as well.

This will happen when we have an expiration, or a split processing, where the strikes or expirations have changed and therefore the options will have changed. Right now, the server clients don't have any mechanism to remove instruments - the users have to zero out positions in the server, and then place new positions in the server. It's a hack, but born out of necessity. Today we got the instrument removal on a reload working so that the splits processing will be much easier and setting the stage for the nightly regional roll-overs where instruments will be dropped as a natural course of the week.

I also spent a little time updating the project web page to put in the overview of all the tasks I have planned out for the remainder of the project. It's a lot of little things, but thankfully, nothing seems to be too difficult that can't be solved in a day or two. Oh, it'll still take a month or so to get all this done and tested, but that's plenty of time before the DST switch this fall.

Today I got news that the plans I'd had for having MarketMash cover the Hong Kong user's open reliably were dashed because of the pokey nature of the Chicago folks in getting their trades into the systems. So I had to go back to the plans for a true 24hr Server and finish them. The first ideas I had for covering Hong Kong were to have the server reload itself without shutting down. This would be similar to the reloading all positions I'd worked out a little bit ago - something that used to require a recovery now is done in less than a minute. But the problem turned out to be the timing.

The traders and operations don't get done putting in trades until nearly 5:30 pm, Chicago time. This meant that even if I had a file ready for me then, the earliest I could have had it going was 5:31 pm, which is 7:31 am Hong Kong time - depending on the DST shift. So the hopes of doing it at 4:00 pm were out the window.

The next idea was to have the instruments classified into 'Regions' like ASIA and OTHER/LAST where they are in these regions based on the geographical region and the time we got marks and end-of-day positions set for them. So, late in the afternoon, we'd roll-over the ASIA region to tomorrow's business day and reload the marks and SOD positions. Then, a few hours later when the rest of the marks and positions were set, we'd roll-over the LAST region and everything would be ready for the next day. In the interim time, part of the instruments would be on tomorrow's date, and the others would still be on today's date.

This means that we'd have to make the server's clients (the middle-tiers) smart enough to know what region they are in and how to deal with the different regions and time of day. It's a bit of a mess, but it's the best solution I can think of, because the only other solution I can see is to have multiple servers with the same problem as nothing is going to be completely ready 24 hrs a day. That's just not the way this place works.

So today was spent doing a lot of thinking about the edge conditions and what would need to happen to each system for each case. Once it's all laid out, I'll update the project web site and the others can see what needs to be done and by whom.

I guess that's really redundant. It's because of the environment that we evolve, but it seemed especially true when the environment itself is evolving due to the changes that are happening. Very cyclical. Interesting.

This afternoon Rock Star programmer was asking some totally left-field questions, and I answered them because I knew the answers and I didn't want to appear to be insensitive and not answer him. Then other conversations sprang up, and in the end, it came back to the idea that Rock Star thinks that there are some broken things that need fixing. Now I'm not one to stand in the way of solid improvement, but if things are working - even just mostly working, then there just might be a reason that they are being handled the way they are. For instance, it could be silly, but the way things are being done may be a little broken due to audit or compliance rules. Nothing we can do about that, but if you don't know there's a reason for a certain action or process, then it could appear to be at least inefficient, and in that case, old Rock Star sees inefficiency as a reason to act.

Again, I have to admire his heart. It's in a decent place, he's trying to fix things that, for all he knows, people have been complaining about for months. Problem is, Rock Star doesn't bother to actually find out if there's a reason, or if people are really bothered. He sees the inefficiency and treats it as if it were horribly broken, and wants to attack the problem.

This, understandably, puts people off. For instance, if you have a way to move a gallon, and once a week you need to move two gallons, it's annoying that at that time you have to make two trips, but it's not as if you can't move any water - just not enough to make it in some trips, some of the time. There's inefficient and then there's broken. It's important to understand the difference.

Unfortunately, when we try to explain this to Rock Star he counters with the classic argument: ...why are you all trying to stop me from doing my job? I just want to improve things. Indeed. How can one possibly know what needs improving until one understands the reasons for the current process or problem? It may very well be as efficient as possible - given real-world limitations of money and time. Again, I'm not against improvement, but trying to fight for a 1% improvement in something that's basically working and leaving some other things totally undone is just not a good way to strike a balance.

And like it or not, business is about trade-offs and balance. You can't spend a man-year on automating something like a letter opener. It's about as efficient as it gets. You just have to accept that you have to manually open letters you get in a day. Now if we received millions of pieces of mail, that'd be a reason to improve it, but what we get? It's just not worth it.

Understanding that there are priorities, and always will need to be priorities, is important to being a good developer. You have to understand how much to do, when, and what the payoff is.

With the resignation of the CTO, and for the time being at least, a little uncertainty as to what will happen with his replacement, I feel there are unsettling times ahead. It really all depends on how they handle his replacement and how close a fit the replacement is to his management style. No one is perfect, and so there's always room for improvement, but this guy was so instrumental in setting the tone of this place that I'm wondering how a more button-down collar type so change the environment that the result would be something so completely different to make it almost unrecognizable.

I can certainly see that some folks would like a more traditional CTO - one that kept lists of deadlines, kept the feet of all IT folks to the fire. There's certainly a place for accountability, and making sure that a promised deadline is met or a really good reason is given as to why not. But to go to the other extreme would so adversely effect this place that I'm wondering if they can really afford to do it. Asking a group of people who have worked a certain way for years to change significantly because of a new manager is their right, but I hope they are a little more realistic.

No doubt, there will be folks asking for a more technically savvy CTO. Again, not a bad idea, but the technology is such a small part of really getting these big projects done. It's more about motivation, people skills, and really only a very little about knowing the underlying bits in the systems. Again, a CTO that doesn't understand the problems is a problem, but one that can't motivate the troops is as big of a problem and causes retention issues, etc.

I guess I'm just nervous. After all, they didn't ask me about it and I seriously doubt that they care how it will effect me. I, on the other hand, care a great deal how it is going to effect me.

Well... I'm back from vacation and it's a bit of a challenge getting caught up to things, but handling email from home made things a lot easier - at least on that front.

The exciting work I was doing on the coding serendipity right before I left for vacation took most of the day, but boy was it worth it. The brute force method of clearing out the positions and reloading them from the files was taking a lot longer than I had planned. Like minutes versus seconds. But with a little work and thinking about the problem carefully, I was able to get then entire process down to about 20 sec. It was very nice to see it working just as I had hoped. This is going to really save time when compared to the 20 min. recycle time if I had had to restart the server.

On a sad note, the CTO resigned while I was away. He'll stay on for a while, but he has always been a real asset to this place. I have to admit that I'm more than a little worried that things won't be as nice around here without his steadying influence, but there's nothing I can do about it, and I know that in the end, things work out, and life goes on.

Today was my first day back from the long Macalicious weekend where I got the two MacBooks for my oldest kids. My son (13) loves the games and is saving up for some game he saw in the Apple store. He's playing DVDs and surfing and loving it. My daughter (11) is having a blast with iLife (iWeb, iMovie, iTunes, and iDVD) and ready to publish a new web site with movies and pictures any day now.

Work is a little tough getting back into it. Lots of emails to catch up on... needed to spend an hour or so with a new guy to bring him up to speed on the components of the system I've been working on for the last 6 years - he's a new second-shift Ops, and will be needing this information as soon as he can get it. Then there was a bug to file with the vendor of our reporting framework, and a few other things to play catch-up on.

But probably the most interesting thing that happened today was the Slashdot article about the Google Maps image of the prototype Chinese nuclear sub. This is incredible, and at the same time shows once again that the cost of destruction of weapon is far far less than the cost of it's construction.

I think the first real point of this was the Falklands Islands War where a missile took out a large British warship. The British won, but the cost of the loss of the warship far outstripped the cost of the missile it took to take it out. This lopsided equation is now even more clear with the Google Maps realization. There have always been spy satellites, and yet the problem with having them is that you can't really tell people what you see as then they know the capabilities you have. But with this, Google plays a somewhat disinterested third-party and "outs" the Chinese sub without the western powers having to leak that the sub exists. Amazing. Just like that, the western governments can bring up the boat as a "problem" without having to expend any of their intelligence in the bargain.

The web and companies like Google are going to change the way people look, act, respond, and are governed. Wild.

There are days when things work out nicely. You're able to clean up a lot of little things that have been sitting around waiting patiently for you to have to the time to get around to them. You hope you do, but many times it'll be weeks if not months before you can really get the kind of time you need to finish off these things.

And then you have a day like today.

Calm... no production problems... plenty of time to go through all those little Post-It notes and put the documentation about building or setting up this app... clear up the comments on this class... make sure the latest versions are available for the docs. It's nothing that's going to make or break a deal, but by taking the time to do it, it's going to make it easier for someone coming in later to pick things up.

Not quite sure if there's another day or two of this, but it's nice to have a little break from the action to get caught up on these things so that I can point people to them and not have to answer their questions in person all the time. Plus, it's there now, and it's not on my To Do list.

Today I think I'm turning the corner with the Rock Star developer that joined recently. Today he came over to say that there's a problem in a class collection that I had put together. Basically, there's a general interface to a table and both the table and views into tables implement this interface. That means that they can be "stacked" and one view can be placed on another to make a rather advanced view on the base data table. The key is that the tables are like database tables, and the views are like database views, so in some cases - like an aggregated view, it's impossible to set a value of the underlying data because it's aggregated. Same with database views.

Anyway, Rock Star wanted to say that the setValue() method worked in some cases and threw an exception in others and that was dependent on the implementation of the general table interface. He was saying that this was a bad design. I told him that there were methods in the interface to see if you could set values on this instance, and that if the method returned false, then not try to set them as it was going to throw an exception.

He tried to say that this was an inconsistency. "Not at all", I said. It may not have been how he would have one it, but there's nothing inconsistent about it. He then spent about 15 minutes trying to say how he'd do it - with multiple interfaces inherited from each other to make it clear what one instance was capable of having done to it. I pointed out that this idea would then lead to a ton of interfaces and then you're going to have a ton of methods returning the values cast into the right interface, or a bunch of if-then logic with instanceof tests to see what interface it was.

In the end he agreed that it was his inexperience with the classes that lead him to believe there was a problem. I felt as if we might have turned the corner. It would be nice if he learned that before he comes over to tell me how badly something is done he does the legwork to make sure it's not going to come back and bit him in the rump - just like today. I think he's got talent, he just needs to adjust his attitude to the rest of us and the development/deployment environment we have in place here.

Today has been one of those days that seemed to start off bad and get worse as the day went on. Looking back on it, I suppose it started decently, but that's really only in relation to how it ended. I just want to go to bed and have the day end and put it in the history of my life.

Once of the developers moved to a cube close to me and even before he did, he said to me: "We'll fight over the blinds, Bob, and you'll give up." My cube backs to a set of windows and as such, to keep the glare on my monitors down, I've kept the blinds closed for more than three years - ever since we moved into this building. The developer knew this and he still seemed to want an argument. Odd. Anyway, today started with two of the blinds pulled up and while neither caused any glare at the time, it was 5:30 am, and the sun was just coming up. My rule had been: if it's causing glare, then they get shut. Well, I left one of the two open, but a few hours later, the other one was causing too much glare and had to be shut.

Now we get into the "up-down-up-down", Keystone Cops phase of the program. What's funny to me is that he had the choice of where to move and he could have moved to a cube that's got a real view of the windows on the other side of the building, but he didn't want to walk the extra 20 feet to talk to some people. Guess he really wanted to fight about it... or was just plain lazy. Or both.

Finally, the one blind closed, the other open, which is acceptable for me, we get into the it just keeps getting better portion of the day.

One of the new developers, the Rock Star I've mentioned in previous posts wanted me to look at his code and tell him what I thought of it. I said I would, and he wanted to do it right then. OK, I guess so. What I saw was something that I would have scrapped right away: no constructor, inconsistent use of the imports, virtually no comments, completely optimistic coding... in general, an example of what not to do for a production system. I started going through what I'd have done differently, and he wanted to argue about it.

I said "Hey, you asked me, I'm not interested in arguing about this."

"We're not arguing, we're discussing differences of opinion."

"Well... I don't want to discuss it. You asked me for my opinion, remember?"

So I continued. We talked more about his refusal to write code that was more robust, better commented, etc. In general, I came to realize that he really was committed to this way of writing code even-though he knew better.

At the end of the review he asked: "So, now are you going to let me code on BKit?"

"You've got to be kidding." I said, and went on to say that I can accept that this is the kind of job that he's going to do, but that I don't have to have him work on the projects I'm responsible for. I suspect that this is really what he was after by showing me his code, but it didn't turn out that way.

Then things got ratcheted up another notch when one of the Data Team guys was editing a price of an instrument while another was reloading it in the server. This ended up causing a deadlock - yeah, my mistake for not thinking of the possibility, but then the server had to be restarted. This was at 2:40 pm, and the restart takes 20 mins. The traders were not happy. I can completely understand, but I do wish they'd lay off the verbal abuse when things aren't going their way. It's not like I purposefully took down the server... it was a logic hole, and I've got it fixed up, but there was a lot of verbal abuse about missing the last 20 mins of the market. I hate it when that happens.

After things were up, I had about 30 mins to try and fix the bug so it'd be in development for tomorrow, and then Rock Star decides to ask me questions like: What java library is most used? and then Do you believe that java collections are widely used? I was in the middle of fixing this bug and so I answered him as quickly as I could, only to really find out what he was after a bit later.

He had looked up the source for Vector in the Java source and had seen that it didn't use one of my top-three code standards: single-entry/single-exit. He was using this as an example to say that if I thought Vector was "good", then I should think his code was "good" when he didn't code any differently than the Vector code.

What amazes me is that he was still on this kick of me liking his code. I finally got fed up with him and said "Well... when I need you to write code for me, I'll have to accept the best you can do. But for now, I don't need you to write code for me." And I can't imagine that I ever will.

I had to leave early for my daughter's birthday, and the commute was horrible, only to get home and find that work had called. Again, it seems like it kept getting worse and worse as time went on. I just wanted to go to bed.

Interesting thing happened on Friday, and I'm just now getting around to writing about it. One of the new developers here asked me about my opinion on the checked versus unchecked exceptions - otherwise known as the runtime versus typed exceptions. So I talked to this guy for a while about where we use one versus another and why we try to stick to a scheme given that Java itself makes this pretty hard to do. For example, Java has runtime exceptions and typed exceptions in it's code, so when you try to write code like Java's you are stuck trying to make it look like Java, but since there are really no hard and fast rules as to when to use one versus the other, it's a bit tough, and you have of make a lot of judgement calls. Nothing horrible, but something you have to be aware of.

So as we're talking about this, the guy tries to show me that I'm inconsistent in the use of these in the code I've written. I understand exactly what he's trying to do (though he denies it) - he's trying to show me how smart he is in an effort to gain my respect. Missed the boat entirely.

This kind of "See how smart I am?" crud does the exact opposite for me. Sure, you look at the Vector class and there are a lot of exceptions that are unchecked - ArrayIndexOutOfBoundsException is one. In JDBC this would be a checked exception for sure, but here, it's not. Why? Who knows. More importantly, who cares? When you're writing JDBC-based classes and extending them, you have to live in that little sub-universe of Java, and stick to their pseudo-standards. In the case of the Vector, you have to stay in the world of JDK 1.1 and realize that in the beginning they weren't using a lot of checked exceptions, so you have to make you code look a lot more like that - certainly if you're subclassing Vector (which was the case in this instance). Then he asked me to look at his code and what would I do in this specific case.

I looked at the code and said that he should check the input parameters, check all returned values, and single-entry/single-exit all methods. These are the three most basic coding standards that I have and believe in them without question - regardless of the language. He then said that this code didn't need that. A light then went off in my mind and I knew exactly who this guy was. By the way, he was the Rock Star of the previous few day's posts. This guy was a lazy Rock Star coder.

I asked him my one of my favorite questions about self-image and human nature: If you were driving on a road at 2:00 am and came upon a stop light that was red, and could see far in all directions, far enough to see that no one was coming, would you run the red light? His answer was "Yes", as I knew it would be. I pointed out to him that this tells me he believes himself above the "rules" made by others. After all, the stop light is just supposed to keep us safe, or is it the law to obey them? Yes, it's a law, but many people think it's really optional, and so feel that it's their life, and they can decide what's important and what's not, and they decide that this light isn't worth stopping for.

So we look at his code. We talk more and it comes out that he's built server-side code that has all these checks in it, but he hasn't put them in this particular code because he thinks it's not worth it. I ask him: "Why isn't this section of ten lines of code worth your very best efforts?" He has no answer, and wants to change the subject - which means he knows that I'm right, and that well documented, error-checking code is really better than code that's totally optimistic and undocumented. But he was trying to show me how smart he was, and this is really spoiling his plans. Sad day.

This goes into every aspect of a person's life - you either run it out to first base, or you pull up if you think you might not make it. You either go all out every time, or you sluff it off. You play at your level, or you sandbag. It's personal honesty and integrity. As Yoda said: Do or do not. There is no try. and he wasn't talking about giving your best effort and failing, he was talking about giving it everything you have all the time, believing that this is the last time you'll ever get a chance to do this, or you hold back, take it easy, realize this isn't a playoff game, etc.

I have no interest in living my life the latter of those two. I have to give it everything I have every minute of every day, or I'm just not going to feel like I can look myself in the mirror in the morning. And when I give it everything I have, I can look myself in the mirror - win or loose, and know that the only thing I can do is practice, because there's nothing I could have done to change the outcome - there's only what I can do to change it the next time.