What's it all about, Alfie?

One of the things I read in a recent article talked about the ability of an unregulated group's majority to punish and even victimize the minority because they have the majority and people like being in power, and the majority is power, and without governance, they are capable of doing nearly anything.

Well, today I was the victim of the group of developers here at the shop. Interestingly enough, not all were the victimizers, in fact, it was their manager (whom I do not report to) that was really the instrument of victimization. But it was the fact that everyone played some interesting part in the episode that surprised me the most.

There were the people that couldn't believe it was happening and didn't day anything to try to regain sanity. There were those that thought that the ultimate goal is to have group harmony, and if it meant the pain of one for the happiness of many, then so be it. There were the vocal attackers and then there was the instrument. Very interesting now that I look back on it a bit.

Needless to say, I was not happy about the situation, nor do I think I'll ever look at these people the same way again. I have no use for people that are not pulling their weight, and while I was content to allow them to be on another project and someone else's problem, it's clear that this cannot continue. What will come of this, I do not know. But of this I am sure. There will be no more incidents when I sit and allow myself to be victimized by these people. That's simply wrong.

Today I was in a meeting with a few folks regarding the disaster/recovery (D/R) plans for one of my systems and two of the guys there said the singularly dumbest thing I've heard in months.

"You need to have a D/R plan for you, Bob."

OK... I understand that they are concerned about me being killed in the disaster that destroys our building by an act of God - because if it's another terrorist attack, the last thing traders are going to be doing on Day 1 is trading from home. The bloody exchanges will be closed, the airlines grounded - just like on Sept. 11. So now they are talking about the possibility that our building is wiped out by Nature and it's not big enough to really shut down markets. But I'm dead. Because if I can get to a phone, you can bet there are lots of people that are going to be looking to get ahold of me. I get calls in the middle of the night now... they are going to call me if the building burns to the ground.

Just so we have a picture of the level of risk involved.

So for this eventuality that I and not the rest of this place are killed - let's face it, if this entire place is killed, what's the worry? We're not trading because we're dead! So let's say they all survive and I die. Sad, but possible. They want me to plan D/R for me!

I got a brainstorm... how about YOU GUYS do that. How about you guys figure you how valuable I am and put in D/R 'resources' (in this case people and not hardware) and have them 'ready to deploy' (trained) so that should I 'go down' (die), there's a bloody backup. I was stunned. Amazed at their attitude and stunned. Face it guys... if I'm dead I - by definition - don't give a crap about this place any more.

Think about it before you say something so incredibly dumb next time.

Normally, I like the fast-paced environment that I work in, but every now and then it becomes clear to me that there's someone asleep at the wheel. Meaning, they are so focused on the next six weeks (for instance) that they forget to look past that and realize that as soon as this short-term thing is done, it's not going to stop the problem, it'll only make it worse. Take 24hr processing.

If we have one system that's 24hr, then they're going to say "Hey, nice, but we can't blank, and we have to be able to do that!" So solving one problem by making it available 24hrs is not really solving anything - it's causing the next thing to become the 'problem'. Then, when they have that second thing, the users will say "Yeah, that's better, but what we want is blank." and it'll continue until everything runs 24hrs a day and had they just stopped and realized that in the beginning, then we could have saved a lot of time and trouble by designing the solution that will make everything work that way.

Trying to do this piecemeal is not the answer. Everyone involved would agree if they stepped back enough to look at the issues really driving these decisions. But they won't. It's enough to say "They asked for x, we'll give them x." and then move on.

So it's going to be a messy year as more band-aids are applied to systems that don't really need them if the systems driving them were working 24hrs, and they'll eventually have to because the users will pick them off one by one until they have them all running that way. But they won't fess up that's what they want, so it's band-aids.

Yeah... it's going to be ugly for a while.

I had an almost surreal experience this morning. To set the stage a bit, I know that good developers are seemingly harder and harder to find. It's been tough given the list of folks we've been interviewing. Now layer on the Rock Star I've talked about in the past and you have to wonder what can really be done about this?

So here's what happened...

I got a chat from Rock Star about the possible values for a field from a data provider we use for Market Data that is pretty much known for it's lack of documentation in the industry. I've worked with this provider for several years, and the people you deal with are nice, and want to help, but the documentation you get is just very bad. No two ways about it. But given that, you learn that the best way to find out what a field is, is to call it with known (expected) output values and see over the course of a few dozen tests if it's what you think it is. If it looks like you're on the money, then you set up bigger tests, and if it holds, you can assume that it's what you thought it was. It's not efficient, it's not even fun, but when you have little documentation and a phone call to the provider yields "I dunno... have you tried it?", you learn that it's better to figure some things out on your own.

But not for the Rock Star, oh no...

"Where's the documentation for these fields? What are the valid return values?" he asks.

"Here's what we have, and that's all there is." I reply.

"That's unacceptable."

OK, that may not be what you want, but you'll accept it because that's all there is. It is, by definition, acceptable, because that's all they are going to give up. I had to actually laugh at Rock Star and ask him Who do you think you are?

I ask him if he's tried the field on a few instruments to see what he gets back - explaining that this is exactly what I've had to do in the past for anyone that needed to know what was what through this API.

"That's unacceptable."

I can't believe this guy. I'm giving him the way to answer his own question and he's refusing to do it because he thinks there has to be a better solution. I get a little upset and simply tell him I'll talk to the provider and see what they say, but that's it. He starts to say more and I simply answer with "Stop talking and walk away." I was getting upset, and didn't want to get to the point that I'd be unprofessional with this guy even though he was clearly being unprofessional in his refusal to even try the approach I suggested.

So I have to wonder - Is this what my beloved industry has come to? Are we to the point that prima donnas, Rock Stars, are the best we can get if you want someone with any real experience? What's happened to all the people that are good at their job, not interested in using new technologies simply for the sake of padding their resume, and are willing to hunker down and solve the problems as they present themselves - not giving in and working with what's available? Are they all gone? Or are they just so well established in their current positions that the only real turn-over is the remainder?

I don't know, but I certainly need to be smarter about cutting off Rock Star when the conversation degenerates to what it was this morning. It's not helpful for anyone. If he's upset with the answers I give him, I'm going to have to tell him to take it to Management. If they back Rock Star, then so be it. But I think I can make a convincing case that me experience in these providers trumps his silly little assertions of unacceptability.

Today there have been a lot of problems with a system that we have in the shop that takes a price feed. It seems that this vendor's custom code to interface their system to the Bank's price feed was having stability problems. Specifically, it's a Java process using JNI that was blowing out of 1.5GB of RAM allocated to the 32-bit process. Having worked with the Bank's price feed for a few projects I know the symptoms of this kind of problem and how to fix it. I'm not going to say this is the only way to fix the problems, but I've tried a lot of things before finding this solution, so I know what's not going to work to a large extent.

So... the vendor throws up it's hands and asks us for help. Earlier, I had sent one email message as I saw so many flying around about these stability issues. I said "Hey, I've got it working, I know it's hard, but there is a way to make this work." The response I received was "Thanks, but we're going to try to save this design and impact the code as little as possible." Normally, I'd agree with them on the minimal change issue, but this time I knew that a minimal change was not going to work. It wasn't a hard change to make - less than a few hours, but it was a fundamental change in the way they were processing the data.

You see, the data is coming on on a (virtually) single-thread calling an onMessage() method to pass in the message containing the data. Because of the way the Bank's price feed is written, you have to make sure that you take as little time as possible in dealing with this message and return control to the calling thread as soon as possible. This means you can't do anything other than throw it on a queue and then have some other thread(s) taking it off the queue and doing the real processing.

So we get into this phone meeting and they start to say what they've done and tried. They quote some timing figures for how long they take on the processing of an event. This doesn't matter a bit. It's how fast you return control to the onMessage() caller that's going to make or break this system. So on and on they go... I finally say "Here's what you need to do..." and outline what they need to do to make it work.

They say "That may work for you, but it can't work for us."

Remember now, they emailed us throwing up their hands for help on the solution to the problem. So this attitude was more than a little shocking coming from the people asking for help. I was only suggesting a way to queue/dequeue the messages - nothing that couldn't be retro-fitted into their code (I had it on a print-out in front of me) in an afternoon at most.

But still they wouldn't take the advice. So I have to say you can give a developer the answer, but you can't make him use it. I know that in the end, they are going to have to use it to get any kind of scale for long-term stability and growth. Right now, they are, as they have been for the years I've been dealing with them - completely inflexible. Great attitudes when it comes to asking for help, eh?

This morning I have finally finished the coding of the SOD Position editor applet and web page. Today it was the applet activation in IE. Amazingly painful. But in the end, it's working and that's all the really matters.

Also, AdiumX 1.1 was released over the weekend and I updated it. The bug fixes look nice, and I have to say, this is an amazing step up from Fire, and I thought Fire was it. Use it every day, all day long.

On Saturday, I received the package from Apple for iWork '08 and put that license code in to stop the 30-day trial and get things set up right. I am still amazed at the level of thought put into Numbers. The increment GUI tools are clever, and completely obvious to use. The graphs are nice - much nicer than the default graphs in Excel. I'm still very glad I got this.

Today I'm going to try and get caught up on all the things I've let slide a few days as I've been working on the SOD Position Editor. Shouldn't take too long - the first thing (catching up on my journal) is now done.

Today I've been doing more coding on the SOD position file editor. I talked about it a lot in the post yesterday, but it's nice to be moving forward - even if it's slowly. The Java GUI tools are powerful, I'll grant you that, but they aren't something that you can use once every year and expect to be really proficient at much of anything. You have to dedicate a lot of time to getting into each class and how it fits into the collective whole to be able to code a good looking GUI without having to hit Google for every other feature.

Also, today is my anniversary - 22 years. Amazing. Going to try and have a little fun this weekend.

Yesterday Apple updated iWork '08 and with the addition of Numbers to iWork, it was something I really had to try. I've been putting off getting MS Office for a long time. I just didn't like the idea of spending money on another Microsoft product, and TextEdit and Mesa did almost all the tasks I needed. But they weren't 100%, and in the end, I still had to do a lot of fiddling with Word, Excel, etc. So I was happy to see that iWork added a spreadsheet.

I got the trial and tried to open up some of the critical Excel and Word docs that I needed to have. For the most part, I need to be able to read and write these formats, so it was very nice that right there on the web sites, it says Pages and Numbers can read and write Office formatted files. Combine that with the fact that the style of these apps is incredible, and you have a really compelling argument.

Then again, it's $79.

Sold.

I will use the trial until I get the serial number in the email, or get the package from Apple in a few days. It's nice to be able to support the Good Guys and at the same time fit in with the rest of the business world on the file formats. Yup... I'm a big Apple fan.

We have been conducting phone screenings for developer positions at work and I have to say, it seems that the really good developers have gone into hiding... maybe the witness protection program, or something. It's amazing. I've tried to think of my old co-workers and realized that most of them are happy (enough) in their jobs - that or they have moved out of the city/state and don't want to move back. It's getting surprisingly hard to find really top-notch candidates to hire.

I'm sure the I'm happy with my current position plays into it a lot, as does the desire not to get management upset after learning that you're in the market for a new position. But really... I'd have expected more than we're seeing.

Well... the search goes on.

Today has shown to me how annoyingly difficult it is to support an application around the globe when the folks on the other side of the world work regular hours. Face it, it means that I have to work unusual hours and long ones at that, and much of the time, there's a day delay because their day ends before mine starts, and starts after mine ends. Very annoying.

On the upside, it's because I want to turn things around quickly that it's annoying. If I was content to let things progress slowly, it'd be a nice change to the day ..OK, sent that email and have a day to wait for a response..., but that's not what I like to do. Additionally, the problems of having people trickle things to you means that you never really get the full picture until you have gone round and round about it several times and find that nothing is forthcoming. Give it to me all at once and then we don't have to wait for these horrible delays.

But then again, maybe that's exactly what they are planning on. Nah... that's too well thought out. The things bugging me now are simply because they don't think more than one step ahead of their own existence. Crud.