What's it all about, Alfie?

Well… I've really held off about as long as I can, and even then, it's probably a bit too long for what I'm doing. This morning I downloaded the Oracle JDK 1.7.0_13 with the latest security fixes for my main laptop. This is a big switch as I'd been holding out for Apple to step back in and really do Java right, but I think they are past that now, and it's up to Oracle to make Java work on Macs, and I pray that they do.

I'm doing more clojure coding, and with that, I really need to have the 1.7 JDK underneath, as there are optimizations that are done in the compiling if it's on the 1.7 platform. Since we're using that on the linux boxes at The Shop, and I have 1.7 on my work laptop, it makes sense to give it up, and switch.

I guess it's like my 17" MacBook Pro… legacy hardware. I'll move to the 15" with retina display for my next machine, and it's a beauty, and it's nice that it's smaller and lighter with the same pixel count, so it's a win - really, but it's a big change from all the years I've had these 17" machines.

Face the future, Bob… it might not be what you want, but it's all you have.

This morning I noticed that we have a new drop of Google Chrome dev to 26.0.1043.0 with some really nice release notes. OK… maybe they are just bug fix titles, with the Jira (?) tracking number, but it's nicer than the SVN commit logs of the past. And I think I mentioned this last time, but it bears out saying again - The update they have done to the page is really very nice. Clean, sharp, nice.

Thankfully, we're still getting a nicely optimized rendering engine so that we're not getting a lot of flash in repainting the same page on refresh. That's a nice touch, I must say.

This morning I put a lot of effort into making the selection of the children from the postgres database faster. There's just no way it needs to be 30 sec. Since I'd already tried the sub-select, I needed to have a completely different approach. Thankfully, I have a lot of ideas, and it didn't take long to find the one that solved the problem.

The problem is that I was always looking in the "wrong direction". I was using IN and joins, when it was really a much simpler problem. Let's find all the demand_id values from the demands table - based on the demand_set_id - which is keyed. This is a very fast request. Then we use that as a cursor to get the locations based on that.

There are a few nice things with postgresql that make this even nicer - like the FOR loop with the select embedded - very nice. Turns out to be really pretty simple:

The difference was dramatic. Now it's about 450 msec for the same load. That's a factor of 60x! I felt such amazement, what a lucky find!

I was doing some selects from a postgres database into clojure using korma, and they were pretty straight-forward for a master/detail scheme:

  (defn locations-for-demand-set
    [demand-set-id]
    (select locations
            (with demands)
            (fields "locations.*")
            (where {:demands.demand_set_id demand-set-id})))

and it was working pretty well. The data was coming back from the database, and everything was OK. But as the database got bigger and bigger, we started to see a real performance penalty. Specifically, the pulling of the locations was taking on the order of 30 seconds for a given demand set. That's too long.

The problem, of course, is that this is implemented as a join, and that's not going to be very fast. What's faster, is a sub-select where we can get all the demand ids for a given demand-set, and then use that with an IN clause in SQL. Thankfully, I noticed that korma had just that capability:

  (select locations
    (where {:demand_id [in (subselect demands
                                      (fields :id)
                                      (where {:demand_set_id demand-set-id})]}))

Unfortunately, this didn't really give me the kind of speed boost I was hoping for. In fact, it only cut off about a half-second of the 31 sec runtime. Kinda disappointing. But the fact had to be related to the size of the sub-select. It was likely 25,000 elements, and doing an IN on that was clearly an expensive operation.

I like that korma supports this feature, but I need a faster way.

Late yesterday I found a slick little javascript-based JSON formatter and syntax highlighter. What I was looking for was a simple and easy way to get the JSON data out of the demand service and display it to the user. The point was to have a clean way for the users to be able to see the low-level data for a specific demand in a nice way so that they can look into problems without having to go to a developer or support person.

The project manager said that we didn't need to have anything fancy - just display the JSON data for the object as it's reasonably self-documenting, and most of the folks could figure out what they needed from that. For the more serious use-cases, we're still going to need to have a more comprehensive dashboard, but for now, this would be something we could whip up pretty easily and solve 90% of the need.

The code was really amazingly simple:

I have the classifier and then the CSS for the individual classes. Then I just have to hit the server based on the demand ID and show the results. Very nice and clean.

Pretty stylish, too! I'm impressed.

I've been having issues with timeouts on a web app that displays certain metrics for the main project I'm on at The Shop. The set-up is that we have a Sinatra app - using the Unicorn server behind an nginx server that's a gateway for the service. I didn't set this up, so I can't really say Why we have the nginx server in front of the unicorn server, as the point of the unicorn server is to handle the load nicely, and it's all on one box, so we're not getting redirection, but for some reason we have this set-up, and we're getting these timeouts.

Initially, I tried to have an EventMachine timer that sent a NULL to the client every 10 sec. This appeared to work, but then we started having issues with the thread it was in dieing. Then we had to put in a check-with-restart on each call. Things clearly weren't getting any simpler, and then I decided to try to convert everything to jruby from MRI, as we really didn't have any good answers as to how this was being deployed and configured. There were just too many holes.

So I reached out to one of the guys that was supposed to be the architect of this deployment scheme. I didn't know who it was, but when I found out I was really quite pleased - it was a guy that I've been interviewing with several times over the course of the last few months at The Shop.

He explained how the deployment was done, where the config and init.d scripts were generated, how to modify their generation, and in short, how to get rid of the timeouts by simply changing a few of the timeouts in the unicorn and nginx servers. So I took his advice, stripped out the EventMachine stuff from the code and then gave it a whirl.

Worked like a champ.

What a difference getting the right help makes. Inside of about an hour I had all the information I needed and had a solution that was cleaner, easier, and more maintainable than I had started with. The problem seems simple: find help, get it, and solve the problem. But too many folks - including me, initially, didn't do that.

Shame on me.

I was talking to a very good friend the other day, and we were chatting about what I thought of the Ruby world, and even the Clojure world, and I had to say that what I've seen of the two languages, and jruby as an implementation, is very nice - but the communities that sprout up around these two languages are really far more destructive than I've ever seen in my 30+ years of development experience. The ruby community especially is almost toxic in it's adoption of the "Lazy Coder". Don't get me wrong, I'm all for the Magic of the Gem, but there's a point when a library - be it boost or a gem, has to expose what it's doing so that the user can see if it's worth using in their implementation.

What I see most often in the tubists I work with is a complete blindness to what's really happening in the gems until such time as they are tracking down a bug, and trace it to one of these gems. Then they either abandon it for another gem, and re-work their code, or they fork it, make a pull request, and carry on.

I think the latter is admirable, but I think far too little thought is put into the code, and therefore the gems, before a bug hunt is underway. This means that if things work - and in most cases, these are simplistic web sites/services so they don't focus on edge cases, and "production" means "usually up, and mostly working", so things typically work with these gems, and the truth of the cost of these libraries is totally unknown to the user.

If I were doing this for fun, that'd be OK. I'd live with it because it's only a "fun project", but when I'm getting paid for this work, I know that performance matters. Heck, everything matters - from the documentation to the runtimes, to the maintainability, to the ease of deployment. It all matters. So you can't pretend that dropping a gem into your app is magically going to do anything. It's going to extract something for it's service, and that something you need to know about.

But that's the rubyist way… to have "magic" gems. You drop this in, and by convention you have a named method/function and all the rest is done for you. I can really appreciate that - JavaBeans was all about that. But it's how it's taken and abused that makes me shake my head. Use it, but understand what it's really doing. Then you can know when it's no longer the appropriate tool for the job.

Anyway… this is never going to change. In fact, I'm guessing it's only going to get worse with time. I'm an old-timer now. A dinosaur that looks to most like I'm more interested in using stone knives and bear skins, but I tell you this - lack of real understanding is the true source of bugs.

Seems the Google Chrome guys have been busy - this morning I noticed they dropped dev 26.0.1386.0 and with it, some great release notes. Seems 'Tanya' is taking over, and she sees these as important. Lots of nice little changes, but what's really important to me is that someone new is in charge, and they care about communication with the people using their product.

Very nice to see!

This morning I checked and Google Chrome dev 26.0.1384.2 was out with the now common lame release notes. It's like they aren't even trying. But hey, it's fast, renders pages nicely, and it works. In the grand scheme, I can live without the release notes if they keep making good software.

Today I struggled trying to get the same keep-alive going in out production server, but because it's running Unicorn and not Thin, and Unicorn is not an event-driven server, it was a lot harder. To be fair, a lot of this had to do with the problems I was having with the deployment of software to the boxes. The Shop has a lot of progressive tools, but the deployment of packages is not among them. So there were several hours lost there.

But the real problem was figuring out how to integrate an EventMachine into the Unicorn server. Thankfully, I was able to find a simple way to do it:

  Thread.new { EventMachine.run }

So simple, but so completely non-obvious.

With that, it started working and I just continued to struggle with the fact that Unicorn doesn't expose a logger like Sinatra/Thin.

These are not the easy tools that folks make them out to be.