What's it all about, Alfie?

Today a developer stopped by to say that one of the graphing classes - the combo graph (bars and lines), had problems when you degenerated it to a bar graph (no lines), and to a line graph (no bars). I checked and he was right on the money. Now one might ask why this is important, but there are times you'd like to just code it up and not worry about the number of lines or bars and have the GUI widget smart enough to just do the right thing. So I dug into the code to see how hard it might be. Interestingly enough, and I probably should have seen this coming, one was a lot easier than the other.

In the VantagePoint scheme, the combo graph is a subclass of the bar graph (hint, hint) and so the first thing I tackled was the case where there were no lines at all. The problem I had was that when I saw that the user didn't specify any lines in the combo graph I simply did nothing, and that's not the best thing to do in this case. What I needed to do was to see that there were supposed to be no lines, and then set that in the graph. Pretty easy.

Then I tried to do the same kind of thing for the bars, and surprise! that didn't work. NullPointerExceptions deep within the VantagePoint code that had clearly undergone some kind of code obfuscation so that the classes and method names meant nothing to me. Finally, through trial and error I came to the idea that VantagePoint was simply not going to allow a combo graph to have no bars. (well.. duh!)

I changed tack and said "Hey! A combo graph without bars is a line graph!" and what I did was to have the applet code detect the 'lack of bars' condition and switch the graph to a line graph and then plot the lines on that graph. Bingo!

I suppose I should have seen the lack of bars on a combo (bar) graph as a non-starter for VantagePoint, but I assumed that if it had either type of variable it would be OK. That was a mistake. The bars are the driving variables and the lines are the 'decorations' on the base graph. Makes sense now, but finding it in the VantagePoint NullPointerException was not easy. But I got it.

Several years ago I got an iBook after not having owned a Mac for many years. When I was at Auburn I was a big Mac Fan because I'd tried to write papers using PCs, and there just wasn't a reasonable set of tools on DOS/Windows that even remotely compared to the tools on the Mac. But when I went into business for myself, I targeted the largest audience possible and that meant PCs. Later, when the only thing that mattered was what I wanted, I got the iBook. I looked around, and while TextEdit was OK for somethings, and ProjectBuilder used it, I wanted a little more powerful editor. I went with BBEdit.

I haven't regretted the decision, BBEdit has been a wonderful tool. But a while back, Hydra was written by some interesting guys in Germany. Collaborative text editing. I could see that being a big deal. They had to change the name, so they picked SubEthaEdit. It was shareware for a while and then they went full commercial with a 30-day trial period. There were some things about BBEdit that I didn't like, and as importantly, I wanted to like SubEthaEdit. So I gave it a spin.

I liked the syntax highlighting a lot better than BBEdit's. I liked the collaboration, though I have to admit that I've never really needed to use it. Probably most importantly, it looked more like a Mac OS X app as opposed to a converted Classic Mac OS app - which is where BBEdit originally started. There were a lot of things that I thought could be better, and my philosophy has been - if you want to ask for more, pay for it. So I did. And then I sent in the requests.

Things got a little better, and now with version 3.0 several things have gotten a lot better. The speed of syntax highlighting is vastly improved. They have added the ability to save an editing session with all the meta-data so that it's easy to pick up where you left off. They also added SSL encryption on the communication so that you can edit in secret. Nice features, to be sure. And while I appreciate that they are not interested in making a bloated package, there are a few things that I'd still like to see:

• Fix the method signatures in the method list - this is a complaint I have with BBEdit as well. Both apps have a touch time getting the method signatures right. This is a big deal when you're trying to quickly find the method you need to work on. If I have a C++ file with 10 different add() methods, it doesn't help a lot to see 10 add() entries in the list - I need to see add(int) and add(double), etc.
• Allow for marks to jump back to - this is one place where Vim beats BBEdit and SubEthaEdit - the ability to set little marks, go somewhere else in the code, do what you need, and then jump back to the place you left. BBEdit is close with it's Marks, and I've added the key-combinations to make it easy to set them, but jumping is still a mouse-driven process. I'd love to be able to use the keyboard alone. SubEthaEdit doesn't have the feature at all.
• Allow Page-Dn with the cursor - most editors allow the user to 'page-down' the file with the cursor by hitting some key combination. BBEdit does this with Option-<Arrow> and SubEthaEdit allows you to page-down, but you don't move the cursor. It's a mouse action to do it in SubEthaEdit.
• Fix the highlighting of -1.0e-12 in FORTRAN code - I submitted this bug a while back and they didn't get around to fixing it. The problem is that for the number 1.0e-12, their C++ highlighter works wonderfully, but their FORTRAN highlighter colors the '1.0' as a number, the 'e-' as text, and the '12' as a number. It needs to be consistent. Also, both their C++ and FORTRAN highlighters miss the leading minus sign ('-') on negative numbers and color it as text as opposed to the numerical constant. It needs to be tagged and colored with the number - not as text.

The thing is, I still want to like it. I want it to get better. So I get the upgrades, pay the money and send in the requests. After a time, maybe they'll see that I'm serious about supporting them and get the few little features that I keep asking for. I can't think of another way to make it happen, and I'm pushing on BBEdit the same way. I have no desire to write my own, and if BBEdit and SubEthaEdit don't turn out, I can go to Vim and be happy enough, but I want a Mac experience in my editor, and one of these guys is going to get there. I hope.

UPDATE: I figured out that the FORTRAN syntax coloring problem was in the Fortran77.mode file that I got from them - but was not a built-in syntax mode in SubEthaEdit. This meant that I could edit it and fix all the syntax colorization issues without having to wait for them to fix them. So I did. I fixed several problems, in fact. In the end, I went to the bug tracking system and updated the bug entries saying I had solved it. Unfortunately, I was unable to upload the changed Fortran77.mode, but I left a note there for them to contact me if they wanted me to email it to them.

Well... this is a pain in the neck, but I'm glad it's solved. The problem is that I was trying to rebuild CKit on my Mac using make and Xcode 2.4.1. It compiled fine, the test programs linked fine against the generated .dylib dynamic shared libraries, but when I tried to run it I got:

    dyld: Library not loaded: /var/tmp//ccr3sui.out
    Referenced from: <path_to_test_app>/test
    Reason: image not found
    Trace/BPT trap

The problem was that the 'tmp' file wasn't there. When I did an otool -L on the application I got:

    test:
      /var/tmp//ccr3sui.out (compatibility version 0.0.0, current version 0.0.0)
      /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.3.3)
      /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.4.0)
      /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)

When I looked at the shared library that I was testing, it had the same reference to this temporary file. I was stumped and blown away. I googled a lot of things and then found a reference that had a similar problem. The solution was amazingly simple and yet it should have been taken care of my Apple's compiler.

The link phase of the build of a dynamic shared library allows for the addition of a few options that shouldn't really matter unless you want to take advantage of the unique features of Mac OS X. They are:

    -install_name libCKit.dylib
    -current_version 1.0.0
    -compatibility_version 1.0.0

where you can make the versions anything you want, but the key to this riddle is the -install_name. By default, the linker is setting it to the temporary output of the link, and not the name in the -o parameter. The docs say this is the default for the install name, but it's not. When you use the same value for the -o option and the -install_name you'll see that the output of otool -L changes to be:

    test:
      libCKit.dylib (compatibility version 1.0.0, current version 1.0.0)
      /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.3.3)
      /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.4.0)
      /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)

We can now clearly see the difference. As long as you have this library in the path specified by DYLD_LIBRARY_PATH, you're good to go.

I spent hours on this. I was trying all kinds of things to see where these temp files were. It's a stroke of luck that I found the web site with the reference that I needed. Yikes. Well... now it's here and maybe it'll help the next poor sap that's got the same type of problem.

This morning I got a visit from Ammo and he's guarding my desk now when I take a break. It's nice to know that I've got an angry amoeba with an automatic weapon looking out for me.

Every now and then I give a look-see to Sun's Java web site to see if they have stated when they'll start to deliver a Java plug-in for x86_64. There's a bug report on Sun's web site, with a ton of comments blasting Sun for not getting a x86_64 64-bit plug-in working, saying that this is addressed in JDK 1.7 (Dolphin). I took this for a good sign as I went to the Java web site to see if there was a beta of JDK 1.7 available - and there was.

So I downloaded it and looked inside the jre directory to see that no plugin exists. It's amazing. They say they're going to have it, but then they don't follow-up and actually deliver it. Now, it's possible, I suppose, that this is going to be added in the final release, but I'm guessing not, as that would mean that something is in the final release that was not in a beta release - not very likely.

So I decided to see if this Blackdown 64-bit java plug-in would work, knowing that it was stopped at JDK 1.4.2. Interestingly, their site, http://www.blackdown.org/ comes back with nothing in the web page. I've googled for this site, and it keeps pointing me back to this empty page. There are several sites like this that tell you how to install it, but all their links to download it are pointing to bogus sites. I finally found a page saying that Blockdown is no more. That explains it. I did find a mirror site that had it still, but when I tried it with my FC5 box and BonEcho 2.0.0.3, it crashed. Too bad.

Well... I guess I'll have to wait for Sun to decide that 64-bit desktops are important enough to support. Thankfully, my laptop will have it with the new version of Mac OS X (10.5) due out this month. Good thing I don't have to wait for Sun for that.

OK, I'm trying to be fair and even-handed about this, but this morning I got another gotcha from Java (1.6.0_02) and IE (6) and Firefox (2.0.0.7). It's this thin client development that is 90% of what's needed but when you hit that 10% it's a pain in the rear to try and get around it. This morning I spent several hours doing just that.

The problem came up because one of my servers uses a web client to maintain it's internal data. This is nice in that the previous 'editor' was a Java application and it was a lot heavier than the support staff needed to have. Plus, with the web delivery, it was easier to have these folks support the server from London and home. But there's always a cost.

One of the things I did a while back was to have the users able to edit start-of-day (SOD) positions for the instruments in the master file. In order to do this in a reasonably useful way, it meant that I had to have a Java applet. This was not the first Java applet on the editor, and I knew that there would be IE/Firefox issues and how to get around them to make sure that the applet launched, etc. Nothing new there. What was new was the fact that I needed to send back data when the user hit 'Save'. I'd probably have been happy enough with Java applets if I hadn't had to do that. But I did.

Moreover, I had to send these updates through CGI scripts where the old and new data were arguments to the GET request. Not terribly hard, but when you find that IE still allows JDK 1.6.0_02 to send URLs to a server and get back answers, and Firefox doesn't, then we get into the lovely realm of applet signing.

It's not that terribly hard, but I didn't want to get a certificate that would mean the users would have to manually accept, so I went on a search for the location of the certificate that the web developers have built to enable them to sign their applets. This way, I know the certificate is allowed on all the boxes, and I don't have to worry about that.

Easier said than done, and honestly, it wasn't Java's fault. But the idea that in these days of corporate intranets and apps deployed on them, it would seem reasonable to have the security system say "Hey, if it's on this domain, it's OK". I know that's not necessarily safe, but coupled with the fact that there's security in the assignment of IP addresses and the domain naming, it's very unlikely that someone is going to put a box on the net with the right IP and domain to publish fake jars. Could happen, but it's not likely.

So I had to sign the applet. Interestingly enough, Googling this revealed that it wasn't until a recent release of 1.6.0 that this difference in behavior between IE and Firefox came to light. Previously, the behavior of IE and Firefox was the same in that it'd allow the URL connections from the applet to a machine not hosting the applet. So this might change again in the future. Yum...

The next problem was NFS... and it's likely a linux problem, or a difference in the NFS system we have in the Shop, but it turns out that if you copy a file from one machine to a shared filesystem, there's about a 70% chance that the web server will think this is a corrupted file. So, after seeing that the ClassNotFound exception was on the Java console, I went to the web server and copied the file to the same shared location. This finally worked.

After all this, things started to click and I got all the issues worked out.

Now I know that Java isn't perfect, and it's a lot better than ActiveX, which came before it, and I'll even say that it's a alot easier to deal with than AJAX, but I keep thinking that there has to be a better way. I know that AJAX is making strides, but it's not really a revolution, more an evolution. I'm wondering where's the revolution in thin clients? Where's the easier system of making clients deployable easily and run cleanly. I've seen Java WebStart and its a lot like Marimba - you download the app and run it. Yeah, it makes deployment easier, but it's no different than fat clients.

Throwing together a Java RMI server would be possible, have the applet connect in and send the updates - that'd get rid of the CGI scripts, but it's not going to make the security issues with Java any more manageable. Make it easier to write powerful apps that are easily deployed remotely. Tons of OSs have done it - X, NeXTSTEP, even VNC does this. But these are for something large, like a desktop or a big app. I'm thinking small - like most applet work. That would be nice to find.

I was talking to a vendor consultant that is here at the Shop helping us integrate their product into our systems. Happens all the time, we tell them what we have, they know what they need and we work together to hook up all the hoses and pipes so that their product fits as smoothly as possible into our systems, and in the end, becomes yet another one of our systems. So I was taking to this consultant letting him know what I did, as his product had to fit into my systems in a very smooth way.

We talked for about 2.5 hours... him asking me what my products were, how they worked, where the data flows were - nothing unusual. But then he asked me how many people were working on all this. I told him that there was a Team of seven on this part, and a few guys on this part, and there's a Team in the other part of the Bank for this part, but the rest of the whiteboard was me. He was shocked.

Which lead him into the question of what makes a really high-performance Team. Seeing as how I'm here to make this integration project succeed, I humored him and let him lead the conversation. His background, it seems, is in leading large-ish group efforts within the very large vendor's suite of applications. He said he was constantly trying to figure out what made Teams really special. Why did some groups seem to just walk in the light, and others were stuck constantly trying to stay off the Dilbert comics.

His take was experience and clarity of focus. He believed that if you had really experienced guys that knew the problem domain, who could look at an issue and easily distill it down to it's elemental key components, then that Team was going to be a winner. And the problem was having to deal with all the customer support and that meant that lesser talented developers had to deal with those issues. That lead to the problem of morale, and that lead to the fact that in any vendor group, 20% or less to the work and 80% are always the focus of the manager's efforts to make them productive.

I told him I thought he was all wrong. I also told him that if he had 80% of his developers that weren't contributing like the other 20%, he should fire them. And I went on to tell him why.

Maxim #1: People Rise to the Level of Your Expectations - as an educator both in one-on-one sessions and in small and large classes - both in the lab and the classroom settings, I believe this with all my heart. If you set your expectations high, people will achieve them. You have to do your part - you can't make them seem unattainable, you have to assist those willing to learn to get the knowledge, and you have to be fair, honest, and communicate these expectations, but if you do that, then people will rise to that level. I've seen it happen so many times, to me it's a universal truth.

So, for the manager that is trying to get the best out of the 80% - he needs only to expect the same from them as the others, and if those expectations aren't met, then those people have to leave the Team. Maybe they don't need to leave the company, but they need to know that they will not being doing development (cool work) with this group because it simply has expectations of each member that are beyond what this person is willing to do. In the end, you will loose a few, but the number won't be large, and even if it is, the Team will perform better because of it.

Maxium #2: Seek those with Commitment, and Everything is Possible - while many may disagree that this is true I'll put it another, less controversial way: If you find people that can acquire your commitment, they will be able to acquire anything else they need to see those tasks through to their ends. Education, experience, assistance - all these are available to the person with commitment. The consultant I was talking to spoke of experience and the ability to have a clear vision of the task at hand. Those are two qualities of a person with commitment, but those two qualities are not by themselves a complete indicator of commitment.

You can have people who are brilliant and do nothing. You can have people who see, and do not do. True, it's most often the case that people who can do these things are doing these things not because they are naturally good at them, but because they have worked to be good at them - natural talent or not. So while I understand what the consultant was saying, I think he was looking in the wrong place for it. Look for the traits that create and inspire the effects, as opposed to looking for the effects.

And this brings me to the core of the issue: Character. If you have it, you'll go far, and a Team with a critical mass of it will infect the others that might be a little low, and the result will be a Great Team. I've been on Teams where I'm the weakest person, and I worked very hard to not be the weakest person. I strove to be the person that could be relied upon to carry the day, if needed, but most times it was just a friendly competition to see who could do better - today. Tomorrow, the race starts again.

Yesterday I was playing with Skitch and FlySketch and saw that when I tried to do a screen snap in Skitch with the hotkey Cmd-Shift-5, I got a lot of problems and then didn't get the snap I wanted. The screen seemed to freeze, I couldn't get it to stop, and then I hit Enter, and I got a snapshot, but it wasn't what I wanted. I was convinced that this was just a conflict in the two applications - both of which use some kind of hotkey combination for their behavior. Not a terrible thing, but I don't like these conflicts as they sacrifice stability, so I picked Skitch over FlySketch, and wanted to give it a shake.

Well... overnight I thought more about it and wondered if it were possible to get the kind of things I liked in Skitch in FlySketch. So I flipped open the laptop and started looking at the preferences of FlySketch. When I saw that it's full-screen snapshot hotkey was Cmd-Shift-5 I knew that I'd found the problem. So since I have FlySktech set to pop-up on Cmd-Ctrl-F, I made the full-screen snapshot in FlySktech be Cmd-Ctrl-Shift-F - something easy to remember. Then the Cmd-Shift-5 would be Skitch's and there would not be a problem.

Or so I had hoped.

So I fired up Skitch again and along with the newly-configured FlySketch I tried the Skitch snapshot - Cmd-Shift-5. It worked!

Wonderful lesson to learn: Always check for hotkey conflicts.

This morning I received my Skitch invitation. I read about Skitch from one of the new feeds I read daily, and the demo movie looked like it had a lot of nice features for image capture and sharing - though I'd be a lot more interested in the capture than the sharing, but there's stuff to share, and I guess they make their money off the adds they run on the viewing pages. Anyway, this morning I got he email saying I had received an invitation. So I decided to go back to the site and see if I was still interested.

Since I had put in the request many months ago, and I've started using FlySketch more, I wondered if I'd still think it was as "cool" as I did when I asked for the invite. Well... I have to say that it really is an amazing little tool. FlySketch is nice and small, and it can do a lot of things, but the ability to drag from Skitch to an app without saving (they save the image to /tmp and then 'drag' it in) and the ability to do more drawing than FlySketch is capable of makes Skitch a good upgrade from FlySketch.

Then when you look at the ability to put the images on their servers, allowing people to look at your sketches without you having to store them - well, that's just a nice bonus. Now I'm not sure I'd ever really trust a service that's free to keep my important images, but it's a place to throw stuff up there, and then decide if I need to keep a copy on my machine(s) later.

It's integration with OS X is really stunning and the mouse-over help is nice, but I hope as they mature the product it's able to be shut-off in the preferences. It works wonderfully well on my MacBook Pro - fast and snappy. I like the organization of the preferences even if it's a little against the Apple HIG. I have to say they did a nice job. I'll be running this and see where it'll take me.

UPDATE: it's a good idea to not have FlySketch and Skitch running at the same time. I've found some incompatibilities with the snapping in Skitch if you do. Since they both are about the same, I can see this as a limitation. I'm not really happy with it, but I can see it.

Well... as I thought it might, the read-only copy of the instrument master database failed on me last night and while the primary is working fine, I feel it's necessary to be able to find a test case, or condition, where the replicated database fails so that I can give this to the team working on that project and they, in turn, can fix the underlying issue(s). I'm sure the local database admins will be be involved, as they have to be as we don't have that level of control over the servers and the machines. So, mauled by the sharks (from my previous post) I go back into the water trying to find the test case that will highlight the problem.

Last evening, the server was restarted at 5:49 pm, and the symbol set was divided into four groups of 889 underlyings and all four were sent out to the database proxy for loading. Typically, all four will finish within a few minutes of each other, but last night the first one finished at 17:55:12 and the second finished at 17:55:50 - but the third and fourth never finished. When I reconfigured the server to point to the primary, the four finished within 4 mins of each other - as they should. Clearly, there was something with the replicated database that was causing two of the loading threads to sit there waiting for data to come back. The question is, how to reproduce this?

It gets more of a quandary when you take into account that my development server started at 7:00 pm local time and it was fine using the read-only database - all four of it's loading threads finishing within a few minutes of each other. So there's something that's happening to the replicated copy between 5:50 and 7:00 pm that caused this problem, but it was gone by 7:00 pm.

I have a simple web page on the server's editor that allows me to look at the database operations that are being done in the code to see what the data is in the database and what's being retrieved. This has really helped a lot in the diagnosis of database issues like bad prices and missing key values. Yesterday, when we were having problems with the replication and the prices, I did have a few times when this page would not return all the data. Because it's a Perl script, it'd return what it had processed, but it would still act as if there were more to read (because there was), and yet nothing would come back. I'd love to be able to reproduce that for the guys.

Unfortunately, I haven't been able to. I have no tools at my disposal other than the requests I make. I'll keep hitting it throughout the day, but I don't hold out a lot of hope that this is going to point to anything conclusive. This leads me back to the same spot I was at yesterday - do I trust it? Today, however, the answer is different: No. I'll trust it when I have to trust it and not before. Since no one is really as concerned about this as I am, I'll stick to using the primary and see where the chips fall. There's no reason to risk production outages when all I've got to diagnose the problem is a few data loading scripts.