I Love Magic – as Entertainment
I love a magic show. Even one that my friends might think of as lame. I love the well-done illusion. I know it's not real, but it's fun to believe that it is. After all - it's entertainment, and if you don't enjoy entertainment, then watch something else. It's your time, your life, your choice. But where I don't like magic is in languages and coding - there I absolutely hate it.
Take clojure, and for that matter, ruby falls into this category as well. The Ruby-ism of Convention over configuration is a nice thought, and can be helpful for new coders starting out, but it obscures all the details, and in that obscurity, it masks all the performance-limitations, and that includes threading. Clojure is the same. What's really being done? Don't quite know with a lazy sequence, do you? What's loaded when? If it's a database result set, does the code load in all the rows and then construct them as a lazy sequence, or does it read a few rows at a time and leave the connection open? Big difference, right?
So I'm not a fan of this kind of code - except for simple one-off scripts and manual processes. You just have no idea what's really happening, and without that knowledge, you have to dig into the code and learn what it's doing. Don't forget to stay abreast of all the updates to the libraries as well - things could change at any time simply because of the cloaking power of that abstraction.
Why does this mean so much to me? Because there's never been a project I've been involved in in the last 15 years that doesn't come down to performance. It's always coming dow to how fast can this all be done, and how much can be run on a single box, and so on. All these are performance issues, and without the in-depth, continual, knowledge of every library in use, I'm bound to have to make some assumptions - until I'm proven wrong by the code itself.
And what's worse, is that I know to look, whereas plenty of the junior developers that I have worked with simply assume that it's par for the course, and don't even think about the performance consequences of their code. They've always had enough memory, and CPU speed, and if it takes 20 mins - so what? It takes 20 mins! I wonder if they would feel that way if it was charging a defibrillator for their parents? I'm guessing not.
Time is the one limited resource we all have. Waiting is not acceptable if you can figure out a way to reduce or eliminate the wait. That's what I've been doing for the better part of 15 years now: removing the wait.
Starting late yesterday, I realized that we have a real performance problem with the clojure code we are working with. I'm not sure that the code is really all that bad, as it works fine when the database isn't loaded, but when it is - and it doesn't have to be loaded very much, things slow to a crawl, and that's not good. So bad, in fact, that several processes failed last night as it was cranking through a new data set.
So what's a guy to do? Well… I know what to do to make JDBC faster, I just need to know what to do in clojure to get those parameters into the Statement creation code in the project. Unfortunately, there's no simple way to see how to do it. Clojure, like ruby, isn't well documented for the libraries - for the most part. This bites because I can see what I need to set, but not how to set it.
So I'm going to have to wait for our clojure expert to show up this morning and tell him to dig into it until he can give me a list of examples that I can work from. I have no doubt he's capable of doing it, but it's not terribly nice to have to wait for him to walk in.
But that's my problem, not his.
But Boy! I hate "magic" code.