What's it all about, Alfie?

I noticed this morning that Adium 1.3.4 is out with fixes for Facebook, Jabber, and a new libpurple. At the same time, I was surprised to see that Adium was using Mercurial for source control. It's one of the significant distributed source control systems, but I hadn't heard of it being used for Adium. Interesting.

Anyway, it's a nice update, and since I use it all day long, every day, it's something I need to keep up to date.

I was adding a few things to my web app today and needed to implement a simple FIFO queue in JavaScript, but didn't need a fancy class. Thankfully, the JavaScript Array has everything I need. There's already a push() and pop() which make a LIFO queue, and if you use shift() you get the first element in the array which makes it a FIFO queue. Pretty sweet.

But the really cool part was making a queue of complex data structures. Sure, you can make JavaScript objects, and push() and shift() them, but you can make a queue that has multiple queues in it and then in your own push() method, you push the different parts onto the different queues.

Say I needed to have a queue of names and sizes. I could have the code at the top of my page:

  var  myQueue = { name: new Array(), size: new Array() };

and then later have the method:

  function push(name, size) {
    myQueue.name.push(name);
    myQueue.size.push(size);
  }

So that if we need to process what's in the queue you might say:

  function processAll() {
    while (myQueue.name.length > 0) {
      doIt(myQueue.name.shift(), myQueue.size.shift());
    }
  }

This is a really slick little way to handle data structures. This isn't real rocket science, but it's fast, simple, and for my application, it's just what I needed. Make a queue, populate it with event data, and then process them en masse.

Not bad.

Today has been a day of struggling with Visual Studio 2008, C# and VBA scripts. It's not my favorite suite of tools, but I didn't get to pick them, and there was a bug that needed to be fixed, and this was the easiest way to fix it.

There's a Windows app with an exposed COM interface and the VBA script 'talked' to it and set it in a state ready to do some work. Then the C# app hit it and did the work, but the end result was that the numbers weren't right. So I had to find out what was wrong, and why.

To it's defense, Visual Studio 2008 is a decent windows development IDE. It's got all kinds of nice things to make it easier to develop and debug an app, but it's the fact that I've been away from Windows for so long that getting back into it is just painful. There's no gdb and make, and it's C# as opposed to Java or C or C++. So it's a struggle for every little thing.

In the end, I was able to set breakpoints and see that the values weren't what I expected. There was a bug in the VBA script and the value folks thought were defined, really weren't, and so were assumed to be zeros by VBA. This caused all kinds of problems, the upshot of which was the numbers were wrong.

By defining the enum values in the VBA script, all of a sudden everything worked. The C#, the underlying app... it all just worked. All for the lack of a few undefined variables. I had wondered where these were defined, but the answer I kept getting was "in the DLL". I had my doubts, which is why I was able to see this problem, but it's just another reason I'm not fond of Windows programming.

Life would be so much simpler if I could code on my Mac all day long. Or linux... that's not horrible, but it's not as sweet as the Mac.

I've been doing a lot of stuff today on a project another developer built. There are a lot of interesting ideas in the code, lots to use, but there are little things that keep making me wonder why it was put together the way it was, and that slows down what I need to do. An example is that the code throws a NullPointerException for an unknown command, and that's OK. It's the expected behavior. In my mind, there should never be a NullPointerException thrown because it's just too easy to check all the times you'd get one, and make sure that you don't try to message a null. Simple.

This made me think back to several conversations I've had in the past with developers, as well as lots of things I've read over the years about the productivity gap between decent developers and really top-notch developers.

There's clearly a difference between great developers and good developers. Much of it is experience, domain knowledge, and education. But it's not always that simple. Some of the best developers I've worked with did not have a wealth of experience, but they had made the most with the experience they had. They weren't the ones with the most education, but they kept learning and always drove to the root of each problem to make sure they understood why it happened, and what needed to be done to correct it. And I suppose in doing these things, they acquired the best experience in the time they had.

But how they got it, or what their specific qualifications were, aren't nearly as important as how they did the job. That's the big difference.

A great developer is worth far more than a good one. Even the development paradigms designed to make great developers from good ones don't really work that well. Sure, they will make fewer bugs, and the effective developer multiplier might be 1.5, but it's never going to be 10.0 - it's just not realistic. I think of woodworking... using bad, rough, chipped tools in different ways may make the table look better, but it's never going to be the same as using good tools, and the right tools for the job.

While in many cases, it might not seem like it matters, in my experience there are more times that it does matter that management doesn't think it does, than times that it really doesn't matter. Management fools itself into believing that it doesn't matter if this is throw-away code. But it does. Hacks hurt.

In the end, I haven't found the replacement for a really good developer. They understand the problem, how to design the right system - not too little and it can't expand and grow, and not too much so that it's overly complicated for the task at hand. It's a skill. It's not magic or luck. And those that think it doesn't matter are kidding themselves. It does. All the time.

Today has been an interesting counter-point to yesterday. Today, again, started too early, but when I got here, I was working to get a lot of things fixed and into the web app. It got interesting with a few deployment issues, so there was running around the trading floor - that's always fun, and then a long session getting things into the code. Lots of things done, and once again leaving it all on the keyboard.

Exhausting day.

Decent day.

I'm becoming more convinced that it's not what the day holds for you, but what you bring to the day that matters most. Sure, if you get into a horrible traffic accident today, I think you have a decent reason to say you had a bad day. But if, like me, you have all the basic components of happiness, it's what you bring to the party that matters.

This is nothing new to the vast majority of folks, but it's something I need to keep reminding myself, it seems. I seem to have a bit of a lossy memory for facts like this.

I can't complain, really. Face it... some days are easier than others. Some days it seems that you're tired as soon as you get up. You feel a little behind the curve even when you seem to be making progress. It's no fun, but there's nothing you can point your finger at to say this is the problem. Fix that, and the day will be a lot better.

Nope. Some days there's nothing to point to. It's not a fantastic day, and there's nothing to do but slug it out. You have to realize that there are going to be days like this and when they come, you have to hunker down and just ride out the storm.

Today was one of these days for me.

The weekend was decent... nice weather, but I spent a lot of time outside doing yard work, so I'm a little sore, have a few cuts on my hands and arms, and it's not like I got a ton of rest - I just worked outside. Not horrible, but not really like a little 2-day vacation.

Today started too early (it seemed), and it was just a day I'd have rather stayed in bed. Nothing I could point to, just work that needed to be done, and then slug out the day and commute until I could go to Nina's softball game.

I'm glad it's over.

I firmly believe that the best way to do anything is to give it everything you have. All of it. All the time. As they say, Leave it all on the track. Do that, and you'll never wonder if you could have done more. Never. That's a really nice feeling. The problem is that to do this is very tiring. Very.

I've been running full-tilt this week, and it's taking it's toll on me. I'm really tired. I was really tired this morning. But I kept at it. That is one of the really nice things I learned in grad school - you can keep going after you think you can't, you just have to know your own limits.

And today, I have to say, I wrote some great code for my web app because I was running towards a 'finish line' of the QA team getting ahold of it this afternoon. Well... they didn't end up starting to test the app, which in the end, was fine. I needed the entire day to finally fix up the last of the new features. Difficult. But worthwhile.

In a few hours, I'll be rested (a bit) and I'll be able to look back on today and realize that there's nothing I could have done to get more done today. I got an enormous amount done. Useful, nice things - all.

I just need a little rest.

And a nice weekend.

I've been adjusting my web app today and it's been one of the most interesting and frustrating days in a while. It's possible to get reasonably quick turn-around with the web pages, which is nice, and they can look pretty nice, but the problems with the little things are frustrating beyond belief.

I had a problem with an HTML table, and it was a little off on the right end. Just a few pixels, but it was something I didn't want to see in the app, and I was determined to fix it. I ended up spending more than an hour trying to find the issue only to see that it was a bad colSpan referencing a non-existant column in the table. Firefox didn't mind it, but Chrome did. Arrgh.

I also had issues with a table row that was a little too tall. I could not figure out what the reason was, as I looked at all the elements and there wasn't a reason. Then I scanned the code again and found a CSS height: tag and that was fixing the height of an element. Duh.

So it's nice to be able to make the quick changes, but many times, it's a puzzle to see what you've done wrong when the browser makes assumptions about what you wrote and how you wanted to see it.

One of my favorite lines from Jurassic Park is when the new arrivals to the park are having lunch and discussing for the first time the concept of having living dinosaurs walking around at the same time as modern man. All the scientists agree that it's not necessarily a good idea, some going so far as to say it's a horrible idea. The quote is from Dr. Ian Malcolm:

I'll tell you the problem with the scientific power that you're using here: it didn't require any discipline to attain it. You read what others had done and you took the next step. You didn't earn the knowledge for yourselves, so you don't take any responsibility for it. You stood on the shoulders of geniuses to accomplish something as fast as you could and before you even knew what you had you patented it and packaged it and slapped it on a plastic lunchbox, and now... you're selling it... you want to sell it!

So how does this effect software development? Easy. There are tools like Eclipse, and frameworks like Spring that allow decent-to-marginal developers to take such terrific shortcuts that they never really have to have the discipline to really make the system they create stable, reliable, maintainable.

In short, they get to hack their way to something that works, without ever really taking responsibility for the system they're building. Let me give you a case in point, but this is by no means the first time I've seen this type of thing happen.

The one app I've been looking a lot at recently is a Tomcat web system that should be a fairly simple collection of servlets doing data collection, aggregation and serving it up. Nothing fancy, but there are details to the process that are important to get right. The developer didn't choose the traditional approach, but rather decided to use Eclipse and Spring to make it easy to re-wire the application at a later date. Tons of interfaces, hard to navigate the code and find what's doing what, and for an app that should have a dozen classes, at most, this is saying something.

But it's classic Spring.

Meaning that the XML configuration files are equally as important as the code it uses. I've had quite a bit of experience with apps like this and in the end, it's always a support nightmare as the configuration of a real app that does a single, well-defined, and well-designed function does not need to have the level of configuration that a Spring app does, and because of that, there's a tremendous risk of messing things up when you do have to do a slight modification.

Now if this developer had to really design this system without using Spring or Eclipse, then they'd have to see where the data was that they needed, and leverage that. They'd say "listen, parsing XML is OK, but it's no way to get the dozen parameters I need - I'll get those from a properties file - or a database." When you have to do it, you think about what's necessary and what's not. You are forced to design the system.

Now, in the hands of a good designer, I have no doubt that Spring can assist in the development process. But in the hands of someone looking to skip a lot of the work, it's a dangerous tool. You have to understand the tools to effectively use them. Just like Jurassic Park.

In the coming days and weeks, I'll be taking over this application and there are a lot of things that I'll be changing. I might rip Spring out completely. The problem isn't that hard, and the configuration issues are one of the major brittle features of the system. Something has to give, and I'm not convinced that Spring is adding the kind of value that it's meant to.

There's nothing wrong with getting a leg up with tools like Spring and Eclipse, but when using them means that you don't really understand what's going on in the workings of your application, then these tools are doing you a disservice. Any developer needs to understand first, and then work on the ease of development.

There really is no free lunch.

The Shop is very fond of using virtual machines for lots of things. And in many ways, I can see it. They make it easy to clone a machine - even move it if the physical hardware has problems. They are reasonably efficient, and for processes and programs that only need a fraction of a machine, they are a great way to isolate one from another without investing in a complete machine for each task.

But I've come to learn the dark, evil side of them.

Greed.

Yup, greed.

It comes from the people that configure them. Say I ask for a VM, they are going to assume that it's not a "big deal", and give me a single core and a little memory. Not too surprising, most tasks don't need a lot. But when you get one that does do something significant, the "cookie cutter" approach that VMs allow gets a little cramped.

When you need a multi-core machine, get the machine. If it's a VM, then you need to go back and say "yeah, but I need another core", and the ensuing "Why?" is something you have to deal with. It's not horrible, but it's what you're going to end up doing more with VMs than when you get hardware that's typically going to be a little more than you need.

I'll get by... but I know now that I need to be asking for "non-standard" VMs with more than one core. I'll learn.