What's it all about, Alfie?

When I restarted my laptop this morning after getting the Security Updates, DataGraph informed me that a new version had been released - 1.6. I read the release notes, and it appears that the big changes are in the data table and the help system.

The Help system has been taken out of the app and on-line to make it easier to support, and also keep the size of the application down. Makes sense, but it's nice to have some form of help if you're off-net, and this move makes that really impractical. Designer's choice - I can see the pros and cons, it just might be nice to offer an option to download the current version of the docs if you're laptop-bound, as I am, and want to have the docs with you.

The change in the data table looks very interesting. Much more intuitive and should make dealing with data entry a lot nicer. There were a few additions to the rest of the app - a few new pointers, a few updates here and there, but the big things were help and data. Good enough.

At the same time, I picked up the Framework as I still have hopes of putting this into some apps of mine. I just need to find the motivation and the time.

I was formally introduced to Python in the context of a vendor's application. They used it as the glue that binds and controls the individual components of their system as well as the internal language of operation. It's a gigantic python collection. And the experiences I've had with the product, and therefore by extension, the language have been bad. Miserable, in fact. But I'm trying to overcome them.

Python is a very simple language with great power because, as in perl, people have taken a liking to it and have written tons of libraries and add-ons to it to provide all the power the original author(s) and designer(s) didn't have the time or vision to write. Both languages are impressive in that way.

I have to admit that I'm not a fan of a language where control rests in the level of indentation, but I can see the reason for it - uniformity of the look of the code. You can't have the religious arguments about tabs/spaces or where to put the bracket if you have to indent to create the structure of your application. But I have to admire a simple language where the creativity is in finding and using those additional libraries and putting them to good use.

So I'm trying to warm up to Python, and I think I'm going to make it. It's just got this major handicap right now - it's connected with a product and a vendor that I really can't stand, and getting over that is hard.

Yesterday I ran into a roadblock on this one little project I was working on, and I realized that the docs were useless, the data dumps weren't helping, and in the end I needed to talk to the tech support folks. So I wrote up a little message saying in very clear terms what I was looking at in the client, and that I needed to have programmatic access in their development language to those values.

Pretty simple stuff.

I was not expecting what I got back.

The first person responded saying that the values I was referring to were wrong, and why would I want them? Answer: because the user wants me to get them.

But those numbers are wrong... Here... let me show you how to set them properly (with screen shots, etc.) OK... but that didn't tell me how to get them even one they were 'right'

To this last exchange I wrote back saying "Hey, right or wrong, I just need to know how to get the data." Clearly, I wasn't getting through. I thought I'd get the answer I needed. Silly me...

The next reply was from the second level support asking why I needed that data, and what was I going to do with it? Holy Cow, people... is this an Inquisition, or what? I just want to know how to get at the data that I can see with my own eyes in the Client. I can get all the other pieces I can see, why not this one?

"Bob, if you'd just let us know the background, I'm sure we can help you."

Finally, I gave up. I told them I was making a report for the users and needed the data as they have highlighted it as the data they want to see. To this, I finally get "OK, we'll see what we can do".

I was stunned. Why it matters to them what lead me to this place, or what on earth I wanted to do with it was, and is, so totally beyond me I can't even begin to put it into a common frame of reference. I know for a fact that the first and second levels have no idea how to get the data I need, so maybe they are just stalling for time. But to be as belligerent about this as they were is something I never expected from a vendor - Attitude.

So now I know... when I want a piece of data, I have to ask for it in the right way or they're just going to keep at me to ask it in the "right" way until I do. Only then will I get the information I asked for. Needless to say, most of my co-workers thought I showed considerable patience with this interchange. Many were using not-safe-for-work language after the second email.

Just amazing.

A developer stopped by today asking me about a problem he was having using the BKBaseGraph as it was saying it couldn't lock the dataset for the change - a nasty exception. I got his code which had a section like this:

  private void initGraph() {
    try {
      curveGraph = new BKBaseGraph(0, 1, BKBaseGraph.LINE);
      graphPanel.add(curveGraph, BorderLayout.CENTER);
 
      int pointCount = 10;
 
      // display the points in the graph
      curveGraph.resizeData(pointCount, 1);
 
      for (int row = 0; row < pointCount; row++) {
        curveGraph.setData(row, 0, (Math.random() * 100.0));
        curveGraph.setRowLabel(row, "" + row);
      }
    } catch (Exception e) {
      e.printStackTrace();
    }
  }

The problem comes in line 3 when you are trying to create a zero-row dataset. That's no good. You have to have non-zero rows and columns - default to nothing and let the graph figure it out. For example, it's better to simply say:

      curveGraph = new BKBaseGraph(BKBaseGraph.LINE);

and let it go at that. Alternatively, if you know the numebr of rows, as it appears is the case here, you can put that in and skip the resize alltogether:

  private void initGraph() {
    try {
      int pointCount = 10;
 
      curveGraph = new BKBaseGraph(pointCount, 1, BKBaseGraph.LINE);
      graphPanel.add(curveGraph, BorderLayout.CENTER);
 
      // display the points in the graph
      for (int row = 0; row < pointCount; row++) {
        curveGraph.setData(row, 0, (Math.random() * 100.0));
        curveGraph.setRowLabel(row, "" + row);
      }
    } catch (Exception e) {
      e.printStackTrace();
    }
  }

Either way works, but don't put the zero rows or zero columns.

He came back to me and asked: OK, but how do I display no data on the graph? Well, there are methods on the BKBaseGraph that handle that:

      curveGraph.hideGraph();
      // ...
      curveGraph.showGraph();

the first hides the graph behind a blank white JPanel, and the second brings it to the front. This is useful if you have a lot of updates to do, or you simply want to say "no data - no graph." Simple as that.

A developer came and talked to me today about the colors of the default second Y axes when using VantagePoint 4.6.4 (the last one I have that allowed this feature) as it defaulted to lime green. Not the ideal choice, I'll agree. So I dug into the code and realized that in my setAxesColor(Color) method on the BKBaseGraph, I set the colors of the X and Y axes, as well as the second X and second Y axes - but only if it was a Hi/Low/Vol graph. That was the mistake.

So I changed it to always set the second Y and then optionally the second X if it was the Hi/Low/Vol graph. Works great. After that, I got to thinking that it'd be even nicer if it allowed the axes to be set independently, as well as collectively. So I wrote the methods and added the applet param tag decoding to deal with the axes independently, and then collapsed the code into a more manageable set.

In the end, I have a lot better coverage of the capabilities of the graphs, and still defaults to something very reasonable for the quick graphs that will be used 90% of the time. Nice.

This morning I got notice on the mailing list that the MacVim team has released Snapshot 36 to fix the problem in Snapshot 35 of dropped characters when they are input too quickly. Seems the developer was tinkering around with changing the input queueing of the keystrokes, and it ended up dropping things. He fixed that up and also took care of several memory leaks.

Excellent work. I continue to be amazed at the level of integration they are doing with Vim on the Mac. Love it.

This afternoon I was trying to measure the partial, intermediate results of something that we're trying to improve the through-put of on a Vendor's package, and I realized that the data I was looking at was really only good in relation to itself - I'd hit on one of the coolest things in Quantum Mechanics: The act of observation alters the system.

I hadn't run into it like this before, but under independent tests, I know that a singular blob could be processed in a little more than a second. But but placing in the outputting of timing results, I saw the combined time raise to roughly three seconds. That's a factor of three. Then it hit me.

You can imagine the issue: by stopping to get the time, more time was taken. Not a lot, but it added up. Fantastic! Thankfully, I didn't require absolute accuracy - I just needed to know the relative times spent on succeeding steps. For that, this was more than good enough. But it was a kick in the pants to see it happen right in front of my eyes. Sweet!

A developer and I were talking about the problem I was having with the second Y axis on the VantagePoint graphs, and he brought up another point: "On graphs", he said, "the origin is included and that tends to 'flatten out' the data so that it's not as rich in detail as it might be." Basically, if the range of the data on the y-axis was 1000 to 1100, the graph would include the y-axis origin and the graph would be smashed at the top. Not good.

I ran some tests, and I found that on my tests, the graph didn't include the origin. I asked him to send me data to verify, and he wrote back that the secondary Y axis data - minus the second Y axis, was fine. Here's a prime (but simple) example:

Sure enough, on a line graph, the y-axis origin is included. I was testing the scatter graph. So I dig into the docs and it seems that VantagePoint defaults this inclusion of the origin differently for different graphs. What I then did was to add the ability on the graph and the applet using the graph to force the inclusion of the origin for each axis independently... or not. Up to the user. So we end up with:

which is exactly what he wanted. Nice. Now there's the choice and that's going to lead to better looking graphs for the users.

A few days ago, I got an email from the manager of this web site saying that he was starting to use the dual-Y-axis line graphs in BKit for the first time, and he was having a problem. I knew I had them working, but I checked my test case applets anyway. Sure enough, they worked fine on several systems in browsers and in the Java appletviewer. I sent back an email saying I couldn't reproduce the problem.

He got back to me saying it was the version of VantagePoint that seemed to be the issue. If he used the older version I used in my tests (4.6.4) then it worked as well. But if he used the newest version (4.6.6) then he got the following exception:

  Exception caught on Viewer[hash=07207493]'s render thread.
  java.lang.RuntimeException: Not implemented.  Use TwoDimVectorDataSet or
        TwoDimTimeSeriesVectorDataSet
    at com.visualizeinc.vantagepoint.jdk12.jfc.CommonDataSet.getColumn
    at com.visualizeinc.vantagepoint.jdk12.jfc.TwoDimLineChartViewer.getIsOnSecondary
    at com.visualizeinc.vantagepoint.jdk12.jfc.TwoDimPointViewer.bp
    at com.visualizeinc.vantagepoint.jdk12.jfc.TwoDimGraphViewer.b
    at com.visualizeinc.vantagepoint.jdk12.jfc.TwoDimPointViewer.e
    at com.visualizeinc.vantagepoint.jdk12.jfc.TwoDimLineChartViewer.e
    at com.visualizeinc.vantagepoint.jdk12.jfc.VisualViewer.p
    at com.visualizeinc.vantagepoint.jdk12.jfc.VisualViewer.b
    at com.visualizeinc.vantagepoint.jdk12.jfc.VisualViewer.run
    at java.lang.Thread.run(Thread.java:613)

I sent off an email to the VantagePoint technical support guys and didn't hear from them for a day or so, so this morning I decided to see if I could fix it by changing my use of the TwoDimDataSet to the mentioned TwoDimVectorDataSet.

So I started by making that change and recompiling. I then ran through all the graph types I had to make sure they all worked. The first problem I ran into was with the Heat graph. It was all red. I had to track it down to a few lines in the creation of the Heat graph:

    case HEAT:
      g = new TwoDimGridViewer(getData(), true);
      if (g == null) {
        error = true;
        throw new BKDebugException("BKBaseGraph.switchGraphType(int) - a VantagePoint
            TwoDimGridViewer could not be created on this dataset. This is a serious
            problem as nothing can be changed.");
      } else {
        // make sure that the graph knows to color pts & labels
        g.setOption(VisualViewer.oiColorStatus, true);
        g.setOption(VisualViewer.oiColorStatusLabel, true);
      }
      break;

It turned out to be lines 9-11 which had been put there to make sure that the default color of labels and points was red. This works fine with the other graph types, but for the Heat graph, it makes all the squares that color. The solution is to not call those methods and I decided to add in the color smoothing in it's place:

    case HEAT:
      g = new TwoDimGridViewer(getData(), true);
      if (g == null) {
        error = true;
        throw new BKDebugException("BKBaseGraph.switchGraphType(int) - a VantagePoint
            TwoDimGridViewer could not be created on this dataset. This is a serious
            problem as nothing can be changed.");
      } else {
        // smooth the colors from one square to the next
        g.setOption(TwoDimGridViewer.gtSmoothColorGradient, true);
      }
      break;

At this point, I have a nice Heat map again:

But when I tried to change from using the TwoDimDataSet to the TwoDimVectorDataSet I got a different exception:

  Exception caught on Viewer[hash=03840954]'s render thread.
  java.lang.NullPointerException
    at com.visualizeinc.vantagepoint.jdk12.jfc.TwoDimLineChartViewer.getIsOnSecondary
    at com.visualizeinc.vantagepoint.jdk12.jfc.TwoDimPointViewer.bp
    at com.visualizeinc.vantagepoint.jdk12.jfc.TwoDimGraphViewer.b
    at com.visualizeinc.vantagepoint.jdk12.jfc.TwoDimPointViewer.e
    at com.visualizeinc.vantagepoint.jdk12.jfc.TwoDimLineChartViewer.e
    at com.visualizeinc.vantagepoint.jdk12.jfc.VisualViewer.p
    at com.visualizeinc.vantagepoint.jdk12.jfc.VisualViewer.b
    at com.visualizeinc.vantagepoint.jdk12.jfc.VisualViewer.run
    at java.lang.Thread.run(Thread.java:613)

which is saying that it's trying to call CommonDataSet.getColumn but it's getting a NullPointerException. I'm guessing that this is something to do with the way the TwoDimVectorDataSet is used. Interestingly, all the graphs I've checked work fine with the TwoDimVectorDataSet instead of the TwoDimDataSet. So I'm thinking I'm using it properly, it's just something in their code.

I've gone directly to their web site and put in a support request saying all the data I have on this and asking them to please help me. New version... new code... something that will give me back the double-Y axes on a graph like was (is) available in 4.6.4.

I checked, and the Git for OS X group has built git 1.6.0.2 for the installer. Since I've decided to use them for git, it made sense to update. Additionally, 1.6.0.2 is the version that I've built from source for my OS X 10.3.9 box so it makes good sense to move.

I'm still waiting for some time to get gitweb going on my home server, but it's been a heck of a few weeks, and there just hasn't been time. Soon... very soon. And then there's the project I need to start using it... Holy Cow!

I gotta say... I'm really looking forward to using this. I just wish I weren't so bloody busy this month. Soon enough.