What's it all about, Alfie?

Today I've been pushing on the Hemlock project to add in some new functionality before the Q/A Team gets backed up with other releases and the changes in Hemlock are delayed another month's expiration. All this is not really new, but it brings up a point that a friend made about parenting teenagers.

Such is the life of a parent, I guess. It's a challenge of character to not allow your children's idiocy/lack of appreciation ruin your relationship with them so that in 5 years when they're surprised by how much you have learned, you can start having a civil, adult relationship.

And as I was pushing very hard today for these new features in a project that I really dislike a great deal, I realized that this, too, is a test of character and commitment. It's easy to put in the "extra effort" when you're excited about the outcome. It's easy to be the parent of a grateful kid.

But show me how much "extra effort" you give to a project that you never want to use, and would send into The Daily WTF if it weren't for the fact that people at your place of work would recognize the horrible code and know it was you that sent it in. Show me how hard you're going to work on something that gives you a headache just keeping track of the useless unit tests and what goes where.

Yeah, my friend is right... being an adult is a lot about a test of character. I'm glad I can say that I was able to pass this particular test.

I just finished up on tracking down this nasty little bug in Hemlock that has been giving me a little grief with one of the desks. Sure, it was a little difference, and the changes to the code totaled about 15 lines, but those 15 lines were very important to the process.

It's nice to be able to end the day on a high note.

Maybe I don't need that bell.

I'm not a prima donna of systems design, I know I've put together a lot of things that were good for their intended purpose, been pushed to their limits, and then some, and shown the effects of time. Everyone does. But what I have a hard time dealing with is the kind of interaction I've had this morning about something as simple as a DatingService built by the guys here at The Shop.

It's pretty standard stuff in the industry - you want to have a single place where you can get the definitive word on the business dates for any given calendar. You need to use this to calculate times to expiration, maturity, etc. It's static stuff, so it doesn't make sense to have it sitting in a database for thousands of people to read, but that's been done other places I've worked - better to have it loaded in-memory on a box that answers requests for dates, and delivers them very quickly.

But that's about the last good idea the DatingService had. Some time ago, the developers here - or maybe the management, decided that Google Protocol Buffers was The Way, and while there's nothing wrong with using them as a serialization/de-serialization scheme, you really shouldn't force the user to know that you're using them because that exposes unnecessary implementation details to the user.

If you want to make a service - make a Client too.

Have that client take only those parameters needed to identify the version/type/etc. of the service and then have all the calls return either objects defined in the API, or language-standard data types. It's simple to use, your client code can include auto-reconnect, auto-failover, all the things that a client might want to have. You are in control.

If you leave too many details to the user, then if and when you need to change any one of them, you're sunk - all users have to update which is a major coordination issue. Not ideal.

Take this example of a client I wrote to my CacheStation - a Reuters-based ticker plant that I wrote at my last position:

  CSClient    svr = null;
  try {
    svr = new CSClient(CSClient.CHICAGO_PROD);
    if (svr == null) {
      // ...handle the problem
    }
  } catch (BKException bke) {
    // handle any exceptions like missing server
  }

All details about the communication are handled in the constructor. There are exceptions thrown for a missing server, but you don't have to configure anything - the "servers" are defined as parameters in the client code so that I (as the owner of the code) can choose to have the real connection parameters in the code, or in a file, or in a database - and there's no difference to the user of the service.

The client auto-reconnects, auto-retries, and has methods for blocking and non-blocking calls so that the user can choose exactly how they want to get data from the server. As simply, the data is returned in objects that are defined in the package as well:

  Vector     symbols = new Vector();
  symbols.add("IBM");
  symbols.add("GOOG");
  Vector     prices = null;
  try {
    prices = svr.grab(symbols);
  } catch (BKException bke) {
    // ...handle the possible exception
  }
 
  MMPrice   ibm = prices.get(0);
  MMPrice   goog = prices.get(1);
  System.out.println("IBM volume: " + ibm.getVolume());
  // etc.

The point is that the API/Client should be simple, simple, simple. The more complex it is, the worse it is. So let's look at what I'm trying to deal with today.

To be fair, the code presented here is not mine, and had I written it, I'd have broken it up into logical steps, logging the success or failure of each, which would have helped in knowing what failed, but not in the fact that it failed. So here's the code for connecting to the DatingServer written here at The Shop:

  private void initializeCalendar() throws Exception {
    boolean  shouldInitializeService = false;
    try {
      readWriteLock.readLock().lock();
      shouldInitializeService = !serviceInitialized;
    } finally {
      readWriteLock.readLock().unlock();
    }
 
    if (shouldInitializeService) {
      try {
        readWriteLock.writeLock().lock();
 
        this.rpcClient = new RpcMessagingClient(transportEnvironment,
                                                messengerFileLocation);
        HolidayService holidayService = HolidayService.newStub(
                         rpcClient.getRpcChannel(
                             HolidayService.getDescriptor().getName(), ""));
        rpcHolidayServiceProxy = new RpcSyncProxy(holidayService);
        HolidayCalcRequest.Builder holidayCalcRequestBuilder =
                         HolidayCalcRequest.newBuilder();
        holidayCalcRequestBuilder.setHolidayCenter(holidayCalendarName);
        holidayCenterData = rpcHolidayServiceProxy.rpcCall("getHolidays",
                         holidayCalcRequestBuilder.build());
 
        serviceInitialized = true;
      } finally {
        readWriteLock.writeLock().unlock();
      }
    }
  }

I don't think I could imagine a more Rube Goldberg scheme for connecting to something as simple as a DatingService and getting a list of holidays for a given calendar name. And this doesn't even include the dozen lines for parsing the output of the RPC call which is another serious problem.

Why do I need to know about the RPC? Why do I need to make builders that serve no other reason than to get the data? It's a mess, a joke, and that people think this is good is just depressing.

And so we have my continuing problem with this service - when I was starting this application at 5:45 am, it was fine. The users wanted to have it running nearly continually, so I moved the start time to 00:45, and there, it's not working. Sure, the code isn't that great - goes the RpcMessagingClient need to be shutdown if the constructor succeeds but the other calls fail? The code ends up getting run make times, making a lot of connections with the RpcMessagingClient and then we run out of sockets and the web server is useless.

But if I restart it at 5:45 am, it's all fine and there are no errors.

So clearly there's a problem in their end of things, but with the code written the way it is, I can't give them any more information than the sequence of steps failed. I would think that's enough. But it's not.

No, they are pushing back saying it's fine on their end and that I need to provide them with the detailed error logs and stack traces. WHAT? I have to provide them with the reasons their code is not working? What's wrong with this picture?

Clearly, this place has a lot of dysfunction and a good deal more issues. I am playing by the rules my manager gives me, and if that means that the early morning hours of the web server are shot because they can't fix their stuff... well... I can't help that.

I give up.

This morning I noticed that the 'dev' feed of Google Chrome is at 5.0.335.0, and so I decided to download it and give it another try. The reason I'm not using it more is that I've found it to be very unreliable in connecting to the sites/servers that I want it to. I wanted to use it as my back-up browser, which is Firefox 3.6 now, but it just wasn't as stable and useful as Firefox 3.6.

While I use Google Chrome all the time in my work at The Shop, and find it to be a very respectable browser for the work I do there, the Mac OS X port is just not up to that level of quality - yet. Well... today when I tried 5.0.335.0 it was acting much better about getting to the sites and moving between them. Not bad.

Given this, I may give it a day or two and see if it holds up to the same load that Firefox does. If so, I may switch over. Would be nice to see.

When I thought my cow-like appearance to my manager couldn't possibly get any worse, today happened, and I just feel like starting to wear a bell. No joke, I had no idea how bad it could get in the span of a day. It's stunning.

Recap: When Two Days is Too Long

Previously on The Farm... yesterday my manager stopped by and asked me to implement this new feature into my web app. It was a reasonable request, and I didn't think it'd take too long, but it wasn't just a configuration change, either. Basically, I'm using the H2 database as an in-memory database for a Tomcat web app and it's working pretty well for me. When I was asked a while back to add in alerts for conditions in the data stream, it seemed logical to use the database triggers capability of H2 to do the job. That way, I just deal with the stream of data regardless of it's source. Seemed pretty slick, and it is.

The problem was, this new request was to add the same kind of alerting to a different data stream - also in the H2 in-memory database, but a different table structure. Now as any database guy will tell you, converting a trigger from one table to another one with different columns, etc. is possible, and not too complex, but it's not something you can do in five minutes. Especially when you've got something as complex as the configurable and flexible alerting system I created.

So I said "Yeah, I can do that... sometime between two days and two weeks."

"Really! Come on... you can do it in a day, Right?" he said squinting and showing his displeasure with the answer I had just given him.

"Nope, that's a pretty decent time because you'll want me to have all the capabilities of the other, which means I need to generalize some things, and that's at least a few days."

He left, disappointed that I hadn't said "2 hours", and leaving me with the distinct impression that he thought I was sandbagging, or lying, or at least an ungrateful piece of meat (a.k.a. a Cow).

The Ole End-Around

This morning I was talking to a teammate and he said that our manager told him to ask me about adding this feature to the system. Immediately I was very defensive - did he seriously not believe me when I said it's take two days? Did he think so little of me to ask a guy that doesn't know this codebase one tenth as well as I did? What was going on here? But this wasn't my teammate's fault, so I tried to remain calm.

"Yeah, Steve, I know... He talked to be about it and told him two days. I guess he didn't like that number so he's asking you."

At this point, Steve clearly looks caught in the middle of things. I can relate. He had no idea what was going on as he wasn't involved in the previous conversation, but was hip-deep in it now. So we talk for just a minute or two, and then I decide to pull in our manager again.

"Ralph, I hear you're asking Steve about the feature we talked about yesterday."

"Yeah, we really need it - maybe he can throw it into Hemlock. Maybe he can put it into another page..."

I can clearly see what's happening, this feature isn't important to our group. It's important to another group, but by giving them something they don't have, Ralph looks like a RockStar. Good for Ralph. But he's doing it on the backs of us - he's not capable of doing this, and it's making me madder by the second to think that all he cares about is how "cool" he cal look to the other people here without giving any consideration to his people.

"Fine, Ralph, I'll put it in the other page, and it'll be a highlighted row." And off I went to start coding this up.

So Once Again, You Really Want This?

I code at a fever pitch to get this done because I'm also seeing emails start to fly around about another change coming, and while I didn't have a lot of time to respond to the emails, I did have a few responses to try and limit the work, but at each turn Ralph was saying we needed to do more, not less.

But I got the changes in and started testing them. Amazingly, I didn't have any real issues and it worked just exactly ass I planned. Guess there's something to this "experience" thing. So I then asked Ralph "Ralph, can you get the values from the business for the filter?" as I had in dummy values, but needed the real ones before I pushed the code to Testing.

"OK, I'll go get them." and off he went.

When he came back, he had a little sheepish look on his face and started to explain: "When I first asked Goose he liked the idea, but then Ice Man said it's not going to work because it's not on all the trader's desktops. Sorry..."

Basically, I had wasted half the day. Totally wasted for a feature Ralph was sure we needed to deliver to "look good", and it turns out, it's just a waste of my time.

Man, I was not happy.

But the day wasn't over, while I was doing this work, I saw in the email chain that Ralph had promised that I could get this other work done TOO!

Ralph's Looking Good: Part Deux

I couldn't spend any time worrying about the lost time, I had to get these other changes into two systems and get several things updated and then deploy a few things. It wasn't hard, but it took me several hours to make sure I got it right and that things would work.

When I was about half way done, and saw that I'd have to stay late, I asked "Hey, what's the rush on this? Why does this have to go in tonight?" And no one seemed to have an answer... but Ralph was conspicuously absent. So I had to trudge on...

As I was entering my 12^th hour of work, I finally got things done and checked out. I had wasted half a day on a useless feature, and then hurried through the addition ov several data sources to the systems just to make Ralph happy - more like make him look good to others.

As I was getting ready to leave, I asked Ralph if he had a few minutes... he tried to say he was sorry, but I said "Let's go have a talk", and I walked into an empty office.

What followed was my feelings about his actions of the last few days.

• the feeling of being massively manipulated about the feature addition with Steve
• the feeling that I was lying or sandbagging about the two day time required - and what's wrong with two days? Is it really that much to ask?
• the agreement "for me" as to what I could get done in a day - again manipulated
• the feeling of disrespect, trivialization of my work, and general "Cow-ness" by him

In the end, I didn't ask for his explanations, but he was trying to get them in anyway. I didn't want to hear it. I wanted to state my case and then leave. So I did.

I wonder if I should wear a bell tomorrow?

While I'm not a fan of Flash, I do have to use it in my work with the Google Visualization Widgets, specifically the AnnotatedTimeLine, so it makes sense to stay as up to date as possible. I've been reading a lot about the massive improvements in the 10.1 performance on Mac OS X, so when I heard that Flash 10.1.51.95 was out, I decided to get it.

We'll see if this makes a lot of difference, and until Apple makes Flash really well, I'll need to stick with the Adobe player and ClickToFlash to limit the pain, and then do the few things I really need. It's not ideal, but it's the best I have at the moment.

I saw this morning that NetNewsWire 3.2.5 is out and contains a few fixes for authentication bugs and possible crashing problems. Brent pointed out that he's planning on releasing several small bug fix releases as opposed to waiting for one big one - just to get the fixes out to those effected as soon as possible. Sounds nice.

This morning I saw that Camino 2.0.2 is out with a few security fixes. Sounds about right, but since I don't use it that much, I'm not really that concerned about the threats. Still, no need to be silly and not get the update.

Today I've been working on adding more functionality to Hemlock, specifically, adding a few features to the existing objects to significantly reduce the complexity of the JSP pages that generate the XML for the data. It took the vast majority of the day as I needed to update hours of JUnit tests that had absolutely no relation to reality, but that's been one of my biggest complaints of Hemlock in the first place.

Anyway...

My manager comes by to ask me if I could adapt the real-time alert system I built for the portfolio-level data to the trader's data we also capture. I said "Sure, but it's not trivial, it'll take somewhere between two days and two weeks. Hard to say exactly how much."

"Two days?! Come on, you sure it's not one?" was his response.

"Nope, two days is a pretty minimal bet - the original implementation was weeks, and if we want it to work for portfolios and traders, and then other data streams, well... it's going to take some time." I then tried to lay out what the technical challenges were so that he'd understand them.

"OK, but if you didn't do it right, just crammed it in there - how long for that?"

At this point, I was getting a little pissed at this display. I knew what the point was - there was a new trading system that had no controls on it, and if this alert system of mine could be used to provide that feedback and control then he'd look like a Savior. Sadly, it was going to take two days.

Let's think about this... he's upset that something is going to take two days to deliver. He already knows it'll be rock-solid and work like a champ, and for this, he can't wait two days?

Yeah, right.

It was at that point I realized that he sees me not as a team member, but a piece of meat - a machine for making him look good. I only exist to make him look better in the eyes of his fellow managers in the Shop.

Amazing.

I'm a cow.

This morning I was running and watching a little TV when I saw a Comcast advertisement for their new high-speed service. It seems they are planning on moving to 100Mbps speed in the near future. Now I'm not sure if this would be a great deal for us... we had the Extreme 50 for a few weeks and weren't really thrilled with the speed. It seemed to Liza that it was even slower than the Ultra which is 30Mbps/7Mbps.

I'm not sure if it was the Comcast infrastructure or Liza just not wanting to pay the extra $50/mo (now it's only a difference of $37/mo). But we went back to the Ultra, and she's happy with that.

But I have to say, if I could really get 100Mbps to the house, that would be very sweet.