What's it all about, Alfie?

Today I've spent far more time than I'd like to have on a problem that has seemed to plague me for months. There's a perl client app to my market data server and it is run to get end-of-day prices for a variety of instruments that we don't get (easily) any other way. Seems reasonable.

Oh, how I wish it were...

I've been hearing that it's working, and then it's not. I look into the data, and it's fine. Then it's not. I don't want to take over this application because it's something that the guys should be able to deal with, but it has been getting closer and closer to that point because I keep thinking it's clear, and then there's an issue.

Just yesterday, I had an issue with a wild bug in the market data server. I deployed it and all looked OK. Then today it's broken again. I'm not sure exactly why this is happening, but I wasn't going to take any unnecessary chances: get the latest driver libraries, and make it from scratch. Maybe there's a header file/library mismatch? Who knows, but I was getting tired of this.

I also got their perl application - including the data, and stripped it down to the point that it's getting the data, testing it, logging failures, and then skipping the updating of the database. I've run this and it's returned good data for everything - save the ones where the ticker is bad, or there's not sufficient data in the database to generate a good ticker.

At this point, I'm convinced that the market data server is OK, and the perl interface into it is also OK. The application should run without problem, but if it picks up another mysterious problem, I'm going to be looking at the machine it runs on to see if there's any problems. It's got to be something, and the code just isn't it.

Digging into other people's code and not re-writing it is one of the less fun things I have to do from time to time. It's like a tacit approval of the horrible way they have put together this application. It was (and is) a mess, and should be recoded to be a lot more streamlined and efficient, but that's not my call. Thankfully, for now, it's working. We'll see what tomorrow brings.

This afternoon I found a wild bug in my MarketData server related to the symbology mapping and F/X rate conversion code for my ticker plant. I know this may seem to be a little 'specialized', but finding it was one of those 'Eureka!' moments that comes with a flash of insight.

This particular data provider for the market data server was the ticker plant I'd written to interface to the Reuters RMDS system. The symbology in our Shop for international tickers has an '=' separating the base ticker from the primary country of trading. So Vodaphone in Great Britain is VOD=GB while Google in the US is just GOOG - the '=US' is implied.

These need to be converted into RIC-like aliases for the ticker plant. I say RIC-like because there are synthetic prices in the system created by basket prices and/or mathematical operations done on 'natural' tickers. Say we had an F/X rate in GBP/USD, but we wanted it in USD/GBP - we'd make a synthetic price that's essentially '1/x' where x is the original F/X rate.

Well... it was this '=' that was giving me the problems. When a user asked for a piece of data for a symbol, they could ask for 'native' or dollarized (USD) figures. This means that I needed to have the F/X rate for all symbols at hand for returning the dollarized figures. But if the user happened to give me an alias as opposed to a symbol, I would happily process that and return the values. But there's the rub... if I were given the alias - that happened to include an '=', I'd try to get the F/X rate and that would be impossible.

An example is JPY=.RTL the USD/JPY F/X rate. If I were asked for that, I'd look at is as if it were a non-US symbol with the country code of '=.RTL' when in fact, it was an alias for an F/X rate and as such had no business being converted to dollars. I would try to get this mythical F/X rate (for this F/X rate) and it would be zero - causing the data to be zero.

What I needed was to be more aware of the symbols coming in and to be able to detect which were tickers and which were aliases and not try to get the F/X rate on the aliases. Once I did that, I was golden and everything worked fine. But it was interesting that I was looking at the data and the '=' sign hit me like a ton of bricks. I'm sure glad it did. The fix took all of 5 mins and everything is working fine. Whew!

Because I have had some spare time today, I've been transferring some of the old journal entries from my work journal to a WordPress blog I created for just that purpose. The old system was a single, giant HTML page, and while it loaded quickly, and you could do searches of sorts, it wasn't as flexible as a WordPress system. Which is, of course, why I made the change.

Anyway, it's not all that bad if you get the data formatted reasonably well. Sure, it's tedious, and it's not something you'd do for fun, but if you want to move the entries into the system it sure beats re-typing all those pages. I'm close to getting 2008 into WordPress, but that still leaves 2006 and 2007, and that's a ton of entries.

Baby Steps... just keep taking baby steps.

I just got a little pop-up saying that NetNewsWire had an update for me - to v3.1.7b4. Seems there was a problem at startup, and someone wanted to be able to execute Javascript for the given page from Obj-C. Nothing major, but nice to update in any case.

But after the update I noticed this same problem I've had for a while - every now and then, when refreshing the list of articles, the parser gets "stuck" on PHPmac.com and the "out of" number of RSS feeds climbs rapidly without bound. I typically just cancel the activity, and I'm OK. But I decided to check, and sure enough, the site is there, but I'm not allowed to see anything. I think this is what's causing the problem.

So... if I get time tomorrow I should send in a little bug to the developer and let him know what I've found. I'm not the only one, I'm sure, and it's probably a much larger issue of proper error handling.

[8/20 3:44am] UPDATE: I sent in the bug report. We'll see if they contact me, or an update appears to correct this in the next release.

I'm not a big one for rumor sites, but this rendering of the possible new MacBook is about as cool as I think a laptop can be. While I understand a friend's criticism that the size of the pad could create issues for hands and wrists trying to type, but I think the size issue is a compromise. It probably won't be this big, but it'll be bigger than the Air's trackpad.

I have to say that I like the black border on the screen as well primarily because my background is dark, and this lends to the illusion that the border simply isn't there. This is going to be vastly different for folks with lighter backgrounds, but it certainly hides the iSight camera nicely.

In the end, I'm still waiting for the quad-core, 8GB RAM MacBook Pro with the high-end graphics and HDD, and if I'm lucky, I mean really lucky, I'll get to have something like this with a nice, large trackpad, with gestures, that will make it much easier to do some of the graphics work I want to do. It's like a built-in graphics tablet.

The MacVim Team has cut Snapshot 34 that includes the ability to turn on/off the ATSUI renderer for the app. The ATSUI renderer is the Apple Type Services for Unicode Imaging system - basically, the (old) Apple way for rendering Unicode. It's supposed to be significantly faster for rendering text but in the past it's had the limitation that the mouse events simply weren't supported in the MacVim code. What's happened is they added a superclass that had all the standard mouse support and then sub-classed the standard renderer and the ATSUI renderer. I'm hoping that it's faster, but it's awfully fast already.

I'll be working with this guy for a while and trying to find any problems but it'll be nice if it's faster and not picking up any bugs.

UPDATE: interesting note: I read about ATSUI and it seems it's rather old (Mac OS 8.5) and has been replaced by Core Text in 10.5 (and beyond). So... while it seems we're one step up in MacVim, it seems it's come right at the time when it's really a generation behind. Shucks. We'll see what happens in the future with SubEthaEdit and BBEdit - see if they pick up on this.

I saw that Apple released another update to the iPhone/iPod Touch OS - now to v2.0.2 and so I once again got out the cord and updated it via iTunes. It's really easy enough, but I was surprised today when my laptop tried to sync the updates I'd gotten for a few apps off the App Store - one of them failed and gave a very cryptic error message. I'll have to look and see what the status is after the update.

I'm sure this is hard for a company to become so diverse in it's product set when it's still relatively small. I appreciate they are doing their best, and in a few revisions they'll get it stabilized and things will run a lot more smoothly. I wish they'd get MobileMe worked out - and add SSL to the site. I would like to be able to use that someday, but for now, I'm doing it the old fashioned way - manually.

This morning I remembered that yesterday I was having a few issues getting into the house from the net. Actually, I couldn't. So I wanted to have a look at things before I took off for the train. When I hit the router's configuration page I noticed that there was a firmware upgrade - several minor-point updates, but nothing apparently major. So I decided to update it.

I probably should have thought this through first.

I was on my MacBook Pro. Connected to the router. So I killed the update.

I had to go down and use my iMac which was hard-wired into the network and even then I had to power-cycle the router because it was smart enough to know that it was being managed from another IP address. The problem was the WiFi was dead due to the update. So I had to power-cycle it and then do the update from the hard-wired machine.

Now, had I been smarter, I'd have switched to the other WiFi router before connecting to the first router and doing the update. Had I done that, I'd have been golden. Alas, it was clear right away when I didn't.

In the end, the update was good and the connectivity returned. I'm not really sure if that was the issue, or something else, but the update was necessary and that fixed things.

Today a fellow developer came to me to ask how difficult it would be to make the scatter graph rescale the axes when the z-axis selection changed. This is the BKit graph that has the optional z-axis grouping selection list so that a third column can be used to group the data points defined by the x and y axes for easier viewing.

The problem is that VantagePoint (while a great package) is not universal in it's feature set. For example, the Bar and Combo charts allow for the ability to set the specific Observables - basically, the points in a data set (or Variable). If I could have applied this to the scatter graph, then I'd be set - simply make the Observable visible, or not, based on the state of the z-axis selections and the value corresponding to that data point. But the class hierarchy of the scatter graph did not include this functionality. I'm sure there's a reason, but I with they had included this in all graph types.

So I had to come up with another scheme. The problem is, the obvious one is to have a copy of all the data, and then when any of the selections change, we remove the old data and copy in just the data set necessary to plot the complete two-columns on the graph. The problem there is that's a lot of data moving of which a ton will be unnecessary. So I didn't want to do that unless I absolutely had to.

What I decided to do was nearly as compute intensive, but it turned out to be reasonable in the context of the GUI event. That was to look at the data values of the 'visible' points and then set the limits on the data in the graph to these, and let the graph refresh accordingly. The upside of this is that there's no data moving. The downside is that we're getting limits over and over again when they aren't necessarily changing. There's also the possibility that we might mess up the nice axes selections that VantagePoint does automatically.

Luckly, that last part had already been solved in the base graph class, so when I tried this approach, I was very pleasantly surprised to see that everything worked as it should. Not half bad, in fact. Thankfully, this is only active when the user specifies a z-axis column for the graph, otherwise this would impact the performance of the applet at all times. Not really great. Good enough.

I got this email from my daughter today. It's not often I get to look up and a puppet is staring me straight in the face. Nope... not too often at all. In fact, if I had to bet what would happen to me today, I might very well have laid money on the fact that I did not think I'd get a puppet email today.

Just goes to show you, you need to keep an open mind.