What's it all about, Alfie?

This morning I'm having a tough day. It's not that it's really any harder than every other day - probably, it's that I'm not as able to bounce back from all the little things that are coming my way. The bus ride this morning was bad - starting with a tough exit from the train. I have about 2 mins to get from the train to the bus, and when I have several slow people in front of me on the train and platform, it makes it very hard. Couple this with a bud driver that leaves right away - and then the next bus driver is late, and it gets frustrating.

I come into work and I deal with more of this kind of silliness, and worse, and it gets kinda hard to just say "What the heck".

But I've realized they don't care. Not one bit. So if they want to write code like this - check in things that aren't working, not deploy it and mark it as deployed… Hey… who am I to argue with them?

It's not my place. Period.

So sit down, self. Have a Diet Coke, and just keep your mouth shut.

Yesterday a co-worker re-tweeted this tweet from someone I don't know - but I've heard this attitude so many times in the past three months, it's really quite shocking:

Don't write comments explaining what your code does. Most programmers can read code. Tell me *why* it does it, but only if necessary.

— Bruno Miranda (@brupm) October 15, 2012

This is what I've come to call the Rubist's Aversion to Comments.

I had a discussion with another co-worker today just before he left, and his points were this:

• Ruby coders don't like comments - they see it as "unmaintained fluff" in the code files. That it's possible to maintain the comments as well as the code, seems completely foreign to them, and when I pointed that out, they seem to simply poo-poo it as a silly notion.
• Ruby coders like pairing - his argument is that pairing is better than comments because more people understand the code. When I mentioned that well written documentation serves the same purpose, it was again poo-pooed.

I think the problem is that for the most part, Ruby coders like good docs - like the Ruby Docs, but they don't like to write them. I think this is just because they want to be lazy, and come up with all kinds of excuses to justify the lack of comments. In short, I think it's just immaturity.

When I responded:

@brupm @mikelikesbikes I've learned that after 38 yrs in this business the "Why" is "Because", and you write good docs all the time. Period.

— Bob Beaty (@drbobbeaty) October 15, 2012

his response was:

@drbobbeaty @mikelikesbikes honestly that makes no sense to me. The *why* is, you intention for the piece of code. Not *why* you did it.

— Bruno Miranda (@brupm) October 15, 2012

I get it… there was a time I didn't write comments - but that was when I was writing games for myself and I was a teenager. When I got serious about this profession, I wrote comments. Always.

But that's not the world I'm in these days. I'm dealing with a lot of folks that see comments as a bad thing - certainly something a good coder can do without. After all, if the code is really well done then it's self-documenting (an impossibility for any complex system). So the Holy Grail of a Ruby coder seems to be to never write a line of comments.

I wonder how many of them will think this in 15 years?

I've spent most of this morning slugging through tests to see how the performance of the process is effected by different things it has to do. For example, writing results to Salesforce… and writing processed data to CouchDB… It's not good news, that's for sure, but it's data, and you have to respect what it's trying to tell you - even if you don't like what it's saying.

There's a story here, and figuring it out is the real challenge.

I've got a few ideas on what to test next, but the overall scheme isn't looking good. We use a lot of data, and reading and writing it is just a huge part of the process. The reasons for writing it back to Salesforce are obvious, but a case could be made that this is the wrong direction to take, and we should remove ourselves from a dependency on Salesforce and just make everything stand-alone.

There's a lot I like about this idea - it still has only one copy of the data, it's just no longer in a third-party application that we can't really access efficiently. It does mean that all the supporting tools (report builders, query tools, etc.) that are in Salesforce would have to be duplicated outside in the system we built, but that's a one-time cost as opposed to the continual cost of moving the bits from there to here and back again.

I know that in Finance this would not have been a long-term solution. But we're not in Finance anymore, and I'm trying to go with the flow here and see what plays outside of Wall Street. So we'll see… maybe it stays, maybe it doesn't.

I've spent far too much time messing with CSS for the preview in MarsEdit for my journal. This took way too much time because of the caching of the files and silly problems I made in the CSS file. I really like the preview in MarsEdit, but the caching of the CSS makes it really hard to know if what you're doing is really having the effect you think.

I spent several hours - really, on this. In order to minimize problem, read the CSS from an http:// URL - this seems to have a lot less problems with caching, and then switch back to the file:// URL once things are stable.

In the end, I was able to get it all fixed up, and then copy up the CSS changes to the CSS for the theme I'm using. Not easy, that's for sure, but worth it.

So this morning I'm doing some timing tests to see what the effect is of trying to run multiple jobs at the same time. Initially, we thought that the boxes would be CPU-bound because of all the logic in the pinning and calculations. But that's not the case. I can use top to see that the vast majority of the time the CPUs are idle (2 CPUs in this box).

So what's taking so long?

The monitoring (NewRelic) says it's all I/O - seems reasonable, but then can we speed up the time to make all six runs by putting them all in the run queue at the same time? Maybe it's I/O choked, and if that's the case, then No, it won't help. But if we're latency bound, then it will - so long as the service on the other end are fast enough to handle multiple requests.

It's a lot of questions, so that's why I'm running the tests.

Timing of a particular run is easy - just use time. But if I want to parallel the runs and see the effect in the same manner, I need to be a little more clever:

  #!/bin/bash
  philadelphia &
  central-jersey &
  baltimore &
 
  wait

This is a great little script that runs all three in parallel, and then waits for them all to be done before returning. If I put this into a time call:

  $ time ./parallel

then I'll get exactly what I want to get.

Now it's just a matter of doing all the runs and gathering all the data.

OK, this is all a nice baseline, but what we're really interested in is how they work together. In that case, we ran successively larger sets, starting at the top of the list and running more jobs and timing them:

This looks puzzling, so I'm going to do similar tests - this time serially running the first 2, 3, 4, 5, and 6 jobs and see how they compare:

All this data looks like:

Which tells me that it doesn't matter, really, if we try to run these in parallel. They simply don't scale that way. There's some other limiting factor in the system and it's not the CPU. This isn't terribly surprising, but it's something that means we really need to start looking at these external systems and see where we can limit our exposure to them.

UPDATE: it took 4:10 - just over four hours to do all 40 of the divisions that we need to do nightly. I guess in that light, it's OK, as there were plenty of retries in the run and we still had time to do the Top 40. The problem, I fear, is that we still have a global choke point that we need to work through, but we'll run into that soon enough, I guess.

I really like Netflix and how I can run through an entire series of a TV show that I might have missed when it aired - without commercials, with pause, etc. It's really a great way to watch TV. Tonight, I finished watching Season 1 of White Collar, and I have to say it's a very nice - albeit a bit fluff, but great entertainment. It's the classic buddy story, but this one buddy straddles the line between legal and illegal. The friends are great additions to the cast, and the storied background of the thief reminds me a lot of It Takes a Thief back when I was a kid.

So good. Funny. Sharp. And doesn't take itself too seriously.

Great fun to watch.

I'm sitting waiting for some runs to complete on the UAT box for The Group, and there's not a lot I can do other than to wait. This has been one of those days where I just want to scream - but can't. I've been trying to make progress to feel like I've really accomplished something today, but it seems that I'm getting set-backs at the same rate that I'm removing them. Every "Finally…" is quickly followed by "What's this?!?", and a diversion or blocker that keeps me from actually getting anything really interesting done today.

Looking back at the commit log, I know I did some good stuff this morning, but that's long been forgotten. Now I'm just sitting here, waiting for someone's runs to complete. This is what I really liked about Finance - or maybe just working in small groups: You spend very little time waiting for folks because there's so few things you need from other people.

Not so here, and so I have to wait. I'm not very good at it, but I guess this is going to make me better at it.

I saw a tweet from a friend the other day:

and it got me thinking about a better back-end for Skitch 1.x. I've planned to use CloudApp and just drag things into it and get URLs, and that would work, but it's not as nice as having a great back-end that works with Skitch as it's meant to. That means being able to pull up the history, etc. It's all there. That would be nice!

PinkHearted uses Amazon's S3 datastore to save your images, so it doesn't really cost them anything to store the images - it costs me. That's nicer than CloudApp because they are going to have to mark it up to make a profit, and I can pay Amazon directly. So I decided to give it a try.

Signing up with Amazon is simple, but there's a few steps to it, so it's not as immediate as you might think. No problem, I just needed to slug through it. Then there was the issue of creating the first bucket. In short - an S3 bucket is a top-level directory in your S3 storage account. I simply used Transmit to create one, and it was very simple.

Now I had a simple bucket on S3:

At this point, I could sign up with PinkHearted and then configure my Skitch to use the WebDAV service on PinkHearted as the source. I've got it all ready to go, and if they shut down the Skitch.com service, that's where I'm going to go.

I like that it's all me now, and on S3 it's redundantly stored and should be fine.

This morning I noticed that Google Chrome dev 24.0.1290.0 was out with a respectable set of release notes. It appears they've felt with some crashing issues, but not a lot else. I suppose that's OK, it's still an improvement, and that's a step. I'm not sure where they are really going but it's nice to seem them still moving forward.

Today was another really slow day. There were a few bug reports I was able to ditch this morning, but the rest of the day was slugging through problems that really did need to be addressed. One of the "bugs" from this morning was turned into a feature request, and that was an interesting exercise in frustration.

I've had the talk with a lot of the Rubists that comments are non-optional, but I've gotten quite a bit of push-back from even those that I didn't expect it from. I waited, because I knew a day like today would come.

I was working on a feature, looking at a particularly obtuse piece of Ruby code, and trying to figure out what the author was doing. One of the long-time Ruby devs stopped by to lend a hand, and even to him, this was bad code. Far too "tricky" - just for the sake of being "tricky". Far too compact. No readability - and of course, no comments.

The long-time Ruby dev even said "I hate it when he does this".

I could not have had a better opening for the importance of comments. And he totally agreed.

'Nuff said.

Slugging through this bug was worth it.