What's it all about, Alfie?

I've been adjusting my web app today and it's been one of the most interesting and frustrating days in a while. It's possible to get reasonably quick turn-around with the web pages, which is nice, and they can look pretty nice, but the problems with the little things are frustrating beyond belief.

I had a problem with an HTML table, and it was a little off on the right end. Just a few pixels, but it was something I didn't want to see in the app, and I was determined to fix it. I ended up spending more than an hour trying to find the issue only to see that it was a bad colSpan referencing a non-existant column in the table. Firefox didn't mind it, but Chrome did. Arrgh.

I also had issues with a table row that was a little too tall. I could not figure out what the reason was, as I looked at all the elements and there wasn't a reason. Then I scanned the code again and found a CSS height: tag and that was fixing the height of an element. Duh.

So it's nice to be able to make the quick changes, but many times, it's a puzzle to see what you've done wrong when the browser makes assumptions about what you wrote and how you wanted to see it.

One of my favorite lines from Jurassic Park is when the new arrivals to the park are having lunch and discussing for the first time the concept of having living dinosaurs walking around at the same time as modern man. All the scientists agree that it's not necessarily a good idea, some going so far as to say it's a horrible idea. The quote is from Dr. Ian Malcolm:

I'll tell you the problem with the scientific power that you're using here: it didn't require any discipline to attain it. You read what others had done and you took the next step. You didn't earn the knowledge for yourselves, so you don't take any responsibility for it. You stood on the shoulders of geniuses to accomplish something as fast as you could and before you even knew what you had you patented it and packaged it and slapped it on a plastic lunchbox, and now... you're selling it... you want to sell it!

So how does this effect software development? Easy. There are tools like Eclipse, and frameworks like Spring that allow decent-to-marginal developers to take such terrific shortcuts that they never really have to have the discipline to really make the system they create stable, reliable, maintainable.

In short, they get to hack their way to something that works, without ever really taking responsibility for the system they're building. Let me give you a case in point, but this is by no means the first time I've seen this type of thing happen.

The one app I've been looking a lot at recently is a Tomcat web system that should be a fairly simple collection of servlets doing data collection, aggregation and serving it up. Nothing fancy, but there are details to the process that are important to get right. The developer didn't choose the traditional approach, but rather decided to use Eclipse and Spring to make it easy to re-wire the application at a later date. Tons of interfaces, hard to navigate the code and find what's doing what, and for an app that should have a dozen classes, at most, this is saying something.

But it's classic Spring.

Meaning that the XML configuration files are equally as important as the code it uses. I've had quite a bit of experience with apps like this and in the end, it's always a support nightmare as the configuration of a real app that does a single, well-defined, and well-designed function does not need to have the level of configuration that a Spring app does, and because of that, there's a tremendous risk of messing things up when you do have to do a slight modification.

Now if this developer had to really design this system without using Spring or Eclipse, then they'd have to see where the data was that they needed, and leverage that. They'd say "listen, parsing XML is OK, but it's no way to get the dozen parameters I need - I'll get those from a properties file - or a database." When you have to do it, you think about what's necessary and what's not. You are forced to design the system.

Now, in the hands of a good designer, I have no doubt that Spring can assist in the development process. But in the hands of someone looking to skip a lot of the work, it's a dangerous tool. You have to understand the tools to effectively use them. Just like Jurassic Park.

In the coming days and weeks, I'll be taking over this application and there are a lot of things that I'll be changing. I might rip Spring out completely. The problem isn't that hard, and the configuration issues are one of the major brittle features of the system. Something has to give, and I'm not convinced that Spring is adding the kind of value that it's meant to.

There's nothing wrong with getting a leg up with tools like Spring and Eclipse, but when using them means that you don't really understand what's going on in the workings of your application, then these tools are doing you a disservice. Any developer needs to understand first, and then work on the ease of development.

There really is no free lunch.

The Shop is very fond of using virtual machines for lots of things. And in many ways, I can see it. They make it easy to clone a machine - even move it if the physical hardware has problems. They are reasonably efficient, and for processes and programs that only need a fraction of a machine, they are a great way to isolate one from another without investing in a complete machine for each task.

But I've come to learn the dark, evil side of them.

Greed.

Yup, greed.

It comes from the people that configure them. Say I ask for a VM, they are going to assume that it's not a "big deal", and give me a single core and a little memory. Not too surprising, most tasks don't need a lot. But when you get one that does do something significant, the "cookie cutter" approach that VMs allow gets a little cramped.

When you need a multi-core machine, get the machine. If it's a VM, then you need to go back and say "yeah, but I need another core", and the ensuing "Why?" is something you have to deal with. It's not horrible, but it's what you're going to end up doing more with VMs than when you get hardware that's typically going to be a little more than you need.

I'll get by... but I know now that I need to be asking for "non-standard" VMs with more than one core. I'll learn.

Well... it's been another great day. I added a lot of polish to the second web page of the site, and now it can plot normalized greeks - normalized by the thresholds for the selected Portfolio. It makes them all out of 100, assuming they don't breach the thresholds. It was a nice little addition and I know they are really going to like it.

Also, the support crew finally got the production box working, and it's running just fine. Great news, because now I can update Test to include all the additions I've made to the system - the new JavaScript drop-down menus, the ability to link to the base data I'm displaying, and of course, help and how to contact us. It's going to be great.

Then I'll tag it and write up some release notes so that they can keep track of things the way they normally want to. It's all very fair, I just didn't know I needed these things, so this first release was a little 'sketchy'. The next one will be better.

So a good day, and lots of good style improvements on the pages. It's looking like a real site now. Can't wait to do the next page.

Today I was spending a little time looking for a simple, yet easy-to-use JavaScript drop-down menu code for my web app at work. I have a feeling that I'm going to need it with the number of pages we'll be writing, it's going to make sense to have something like we had back at my previous job. Actually, I want something a lot nicer than we had back there, so I started digging now.

What I found was a lot of JavaScript menu systems for sale. Now I can see the value of commercial software - I just didn't expect it to be this menu. One of the nicer free solutions was this guy I found. It's simple, seems to work just fine, and I should be able to make it work for what I need.

However, there are limitations. First, there's no multiple level menus. Second, the menu structure has to include a bit of JavaScript in it - not horrible, but not as clean as simple UL and LI list items. That's where this commercial offering really makes a difference.

This one takes nothing more than UL and LI lists, embedded within each other, and a few class tags, and builds a very nice menu. Very nice indeed. Different themes as well as different animation techniques makes it a very nice menu.

But it's $200. Not exactly inexpensive when all I really need will be served by the free alternative. I'm going to have to run this by the purchasing folks and see if they'll swing for it. If they do, that would be great. But I can certainly see their point if they say no. After all... the free alternative is fine for what we need.

We'll see what they say on Monday.

I noticed this morning that there was a new version of Coda released a few days ago. The release notes say that there are a few things fixed - including a little better support for Git, which is very nice. Also, there's better syntax highlighting for regexo in Javascript, which is nice, now that I'm getting hip deep into Javascript.

In general, it'd be nice if they also added Tomcat support, but I understand that's not their target audience, and I'm OK with that. It's just such a nice tool.

I was working with the JSON response from my servlet and needed to be able to print the complete structure of a JavaScript object. I wanted to make sure that I was evaluating what I was receiving from the servlet properly. Well... there isn't something that's a part of JavaScript natively, but you can do something simple and you'll get something pretty close:

  var data = eval('(' + xhr.responseText + ')');
  var guts = '';
  for (var key in data) {
    guts += key + ' : ' + data[key] + '\n';
  }

and you get the keys and values for those keys. You aren't not going to get a complete recursive output, but if you wanted to, it's easy to imagine making a print function and you're good to go.

Just a neat little trick.

OK, once again, we have an update from the iTerm group - this one appears to have been release on 4/15 and I just missed it. Fair enough. It's update says:

• Fixed clipping bottom of fonts that are not Monaco (#2735995)
• Smart window placement remembers initial window (#2736180)
• Windows will no longer be placed under the dock (#2740187)
• Fixed default background colour being drawn over background image
• Fixed a bug when pasting more than 1024 characters of text
• Fix overwriting of static window/bookmark titles (#2744363)
• Revert to old force screen update behaviour for 10.4 (#2738986)
• Rewrote large parts of PTYTask

all of which aren't a big deal to me, but nice to see movement is being made. My remaining issue is the lack of "Workspace" support where Terminal.app has it. If it weren't for that, I'd be able to switch back to iTerm. Because I sure do like the absence of the scrollbars on the windows.

UPDATE: I decided to send in a feature request about the 'workspaces' feature. I don't have high hopes for this as it's probably not what they are interested in adding, but you never know. No harm in asking.

I was enhancing my web app today based on a request that someone voiced about making it easier to have a user see everything all the time. Face it, there are a good chunk of users that are in the Risk Management group that are going to need to see everything all the time. If I made a permissioning scheme based on an enumerated list, then when I added a new portfolio, I'd have to update these users. Sort of a hassle.

So the request was to have a wildcard for the portfolio list, and when a user had this, they would be able to see all portfolios regardless of how many there were. It's a good idea, I just hadn't thought of it.

But that's not the issue.

What I realized was that there were likely going to be a lot more changes to the permissioning scheme, and in that case, my semi-colon-delimited list of values was not going to do. What I needed was to be able to pass in a fully created JavaScript object and then let the page be able to interrogate it as necessary.

For example, if my validation applet was given a parameter out of json then it might return the following JSON output:

  { username: 'beatyr',
    page: 'PnLTool',
    approved: true,
    portfolios: ['Indexes', 'Nasdaq'] }

then when parsed, I should be able to say things like:

  if (!userInfo.approved) {
  }

Nice.

All this seems well and good, and I should be able to simply say:

  var userInfo = eval(xhr.responseText);

but that won't work. Why? The fact lies in the interpretation of the initial '{' in the string value. The parser in eval() thinks it's the start of a block of code, not a JSON value, so you have to force eval() to get into object evaluation mode by wrapping the JSON in '()'. Like:

  var userInfo = eval('(' + xhr.responseText + ')');

When you put this into the script, it works like a charm. Perfectly understandable from the point of eval(), but a little tricky if you forget about it.

I got a request for my web app to add in user-based permissioning. Basically, the management wanted to restrict the data any one person in the Shop might see based on their login name. Since I'd already done the user authentication with a signed Java applet, it seemed like a natural extension to add in the portfolios that the individual user could see. I simply change the JavaScript code to parse the response and then went back to the java servlet and added in the database lookup of a new column of data from the database table with the list of usernames.

I decided that it was easiest, for now, to just have a semi-colon-separated list of values, the first would be the YES or NO of the approval, and if YES, then the remainder would be the names of the allowed portfolios for this user. This was taken straight out of the database column.

Like I said, simple.

As I get more requests, I'll probably try to make it JSON and have that allow for the map-like capabilities of the basic JavaScript object. At that time, I'll move to returning JavaScript and then running eval(xhr.responseTest) in the return method. That will make it easier to extend in the future. The only wrinkle is that I'll have to assume that there's a standard variable that the response is coded to, but that's not horrible.

When I got the servlet working, and the simple parsing of the results, I then set about to allow the user to see only that portion of the data that was contained in the portfolio list I'd just parsed. This, as it turns out, was pretty easy.

The first thing I did was to get rid of the repetition in the HTML and JavaScript for all the individual portfolios. There were check boxes, checks on the data, etc. All those needed to be automated and made expandable simply. When I looked at the code, it was pretty simple. Leave all the checkboxes as-is, and create a simple JavaScript array of the names of the portfolios. Then, in the initialChecks() call, use document.getElementById() where the id was the name of the portfolio. Simple.

The tougher part was making the checkboxes disappear when the user shouldn't be able to see their data. The answer was really rather simple: make a div that surrounds the checkbox and name it "div_Portfolio". That way I can make an array of them and get their references by again using document.getElementById('div_' + portfolio). Then, I can make them invisible by:

  // get all the references of the checkboxes and enclosing divs
  var portfolioChecks = [];
  var portfolioDivs = [];
  for (var i = 0; i < portfolioNames.length; ++i) {
    portfolioChecks[i] = document.getElementById(portfolioNames[i]);
    portfolioDivs[i] = document.getElementById('div_' + portfolioNames[i]);
  }
 
  // ...get the list of the available portfolios
 
  // remove the portfolios that the user can't see
  for (var i = 0; i < portfolioNames.length; ++i) {
    if (response.indexOf(portfolioNames[i]) < 0) {
      portfolioChecks[i].checked = false;
      portfolioDivs[i].style.display = 'none';
    }
  }

with this 'turn off, then remove' code, I was able to leave the large part of the app unchanged and it just worked. I was very pleased with the way it turned out.

Now I have something that looks good for all users, and they can see only the data the management wants them to see. Lovely.