Archive for November, 2007

Interesting Argument about Boost Smart Pointers

Thursday, November 8th, 2007

cplusplus.jpg

In my work today on updating the valuation library to a current production version, I once again came across Boost smart pointers. Now there's a lot of good things in Boost, and I have barely scratched the surface of what Boost has to offer, but the implementations I've seen that use smart pointers are much more confusing than they are helpful. After all, the point is to make it easier to write code - not harder.

Well, I was chatting with a good friend about this today after the fact and he uses Boost's smart pointers a lot and doesn't see the confusion. I can agree that if I were using them all the time I'd probably be desensitized to them as well. But I'm not yet used to Boost's smart pointers, and honestly prefer to handle the memory management myself - that's one of the reasons I'm using C++ in the first place - careful resource management. But I see his points, and because I found the conversation very interesting, here's the gist of it.

Take the following little code snippet:


    typedef EQS::Shared<Operation>::Ptr OperationPtr;
    typedef EQS::Shared<Results>::Ptr ResultsPtr;
    ...
    OperationPtr    lOperation = lFactory.getOperation(mType);
    ResultsPtr      lResults = lOperation->execute();
    double          value = lResults->getValue();

While it's perfectly legal code, if you separate the typedefs into the header file and the remaining three lines into an implementation file you have something that's confusing to a traditional C/C++ developer. Are lOperation and lResults pointers or not? Well... in reality, they are and they aren't.

They are in the sense that you can use the "->" to access methods, but in another sense, they are removed when they go out of scope in the code - like traditional stack variables. While this might be seen as a benefit by some, to me it makes the code exceptionally confusing.

In my opinion the line is being drawn too finely. Make theme appear much different from pointers and the confusion goes away. For example, the following code looks odd, and different, but there's no possibility that someone will be confused by the pointer-nature of the "->":


    typedef EQS::Shared<Operation>::Ptr OperationPtr;
    typedef EQS::Shared<Results>::Ptr ResultsPtr;
    ...
    OperationPtr    lOperation = lFactory.getOperation(mType);
    ResultsPtr      lResults = lOperation..execute();
    double          value = lResults..getValue();

or, as if taking a page from Objective-C, this:


    OperationPtr    lOperation = lFactory.getOperation(mType);
    ResultsPtr      lResults = lOperation[execute()];
    double          value = lResults[getValue()];

In both cases, there is a distinct visual difference in how the objects are being used. In these examples, this is silly and trivial, but in large code sections where there is a lot of processing and the usage of the "->" implies traditional pointer, it's confusing.

I know they did this to make them seem as close to real pointers as possible and allow for the auto-CG, but I think they'd be better off - as C++ was with references, in making a new language element. And if they can't have that, then at least overload a different operator - or something to make it looks significantly different from traditional pointers.

I guess my point is that by using smart pointers you'd never think twice about doing this:


    SmartPtr    p = foo.bar();
    p->goof();

but you'd never do this:


    char        p = foo.bar();
    p->goof();

and while you could do this:


    typedef char *CharPtr;
    ...
    CharPtr     p = foo.bar();
    p->goof();

why would you?

Consistency... that's an important part of a language to me. Perl is nice, and yet it's strength is also it's weakness - you can do the same thing a million different ways. But almost all my Perl code does the same thing the same way - just for that consistency.

The conversation has continued and my friend has pointed out that there is a significant historical component to this that I wasn't aware of, and it sheds light on why they are used this way:

fair enough, call it something else, but, history helps shed light here. In the early days of c++, you didn't have any such thing. Then sometime after the first standard release, they said, hey, let's address a simple memory leak issue and added auto_ptr to the stl. Now auto_ptr seems like a reasonable name and they designed it so that you had almost no coding impact to replace your regular c-style pointers with auto_ptr. You could imagine the resistence they would get if you had to re-write huge chunks of code to migrate over to it. Instead, you had to change your type declarations and remove some deletes and that was basically it.

However, people started to realize that std::auto_ptr wasn't so good for various reasons and the boost guys came along and introduced a richer set of pointer objects that kept the same semantics so that it was easy to migrate.

I can see his point, but when he pressed me about what I thought C++ should do I said that they needed to add a fourth elemental data type:

  • value
  • pointer
  • reference
  • smart pointer

and that smart pointer is going to have a different decorator for accessing it's methods and ivars. By making it too close to a pointer, they have in effect made it more confusing.

Where Java Got it Right

After thinking about this overnight, I have to admit that this is where Java got it right. If they had wanted to add in a reference-counted dynamic object in C++, they should have created smart references. Had they done that, it would have allowed for the 'new' without the need for a 'delete'... it would have used the '.' as the method invocator, and it would have allowed for the complete absence of the '*' in definitions. In fact, I suppose that the only thing they are missing now is the use of '->' when '.' would be better. Then, simply call them smart references and the confusion is gone.

You can still have them hold a NULL, unlike traditional C++ references, but that would be the edge condition, as it is in Java, and not the typical use of the datatype. Yup, I have to say that they'd be so much better off in my mind if they went for 'references' as opposed to 'pointers'.

Way Too Much Fun Coding

Thursday, November 8th, 2007

cubeLifeView.gif

Today I took on a project that I was convinced would take me months - the upgrading of our calculation library framework from a very old version (more than 24 months) to a very recent version (in production systems now). I was convinced that going from a v6.x.x to a v7.x.x version was going to be a pain, and plenty of people didn't say I was wrong. But I was very wrong.

It took me about 2.5 hours from start to finish to update the calls and the use of Boost smart-pointers from regular pointers and clean up the code a bit. Amazing! I had no idea it'd be this easy or I'd have done it months ago. Easily. But that just goes to show you that you don't really know how hard something will be until you really roll up your sleeves and get into it.

With this, we'll be able to add in several new features that the traders have been wanting to see, and in fact have just been waiting to use because the older version of the valuation library constantly generated a ton of SegFaults when I gave it the skew and kurtosis curves for the instruments we had. I was able to turn that feature on in the code and it works beautifully. That's nice.

So today has been a very good day for me. Wonderful, in fact.

Problem Setting Full Screen Capture for FlySketch

Wednesday, November 7th, 2007

I thought I had this all figured out, and to a point, I did. But today I noticed a bad conflict between FlySketch and Skitch when trying to do the Skitch 'selection grab' - Apple-Shift-5 hot key. There was a conflict between that and FlySketch's Ctrl-Apple-5 full screen hot key, but I had changed that, as I posted about a while back.

Surprise! It was back.

The problem was that FlySketch wasn't saving the change in it's Preferences. This meant that when I rebooted (like I have to do an update on the OS), I went back to the old mapping that had the conflict. I suppose that I could change the mapping in Skitch, but honestly, I'd like to have FlySketch set to what I wanted, and Skitch to stay where it is.

I asked the question on the FlySketch forum and we'll see what I get back. I'm hoping it's a simple fix - or maybe an update, but I'd hate to have to re-do this every time. I am somewhat forgetful in this regard.

UPDATE: I've decided that it's not worth the hassle, and if they fix it, that'd be great, but until they do, I'm moving Skitch's hot key to Apple-Ctrl-S to match the Apple-Ctrl-F for popping up FlySketch. There's a symmetry there that I can live with.

[11/8/07] UPDATE: I filed a bug with FlyingMeat on this. Turns out it's more likely to be acted upon in this manner as it's email-based and not the forums. We'll see if he responds today.

[11/8/07] UPDATE: I sent along the Console message in response to Gus' reply to my bug that it should, in fact, be stored in the Preferences. It was:

2007-11-08 13:34:14.322 FlySketch[2565] CFLog (0):
    CFPropertyListCreateFromXMLData(): plist parse failed; the data is not proper UTF-8.
    The file name for this data could be:
    /Users/drbob/Library/Cookies/Cookies.plist
    The parser will retry as in 10.2, but the problem should be corrected in the plist.

Later, when I had the time, I removed the entire preferences file for FlySketch from my ~/Library/Preferences/ directory and then restarted FlySketch and put in my serial number. Unfortunately, the same thing happened - the hot key for the full-screen capture was getting set, but it wasn't getting pushed to Preferences. Odd. We'll have to see what Gus makes of this.

[11/12/07] UPDATE: I got a new version of FlySketch from Gus and this fixed the problem right up. Not exactly sure what he changed, but whatever it was - it worked.

Incredible App to Read Medical XRays

Wednesday, November 7th, 2007

OsiriX.jpg

I know I shouldn't be surprised to see technology enter the medical profession, but with a sister that's a doctor, and her husband, and three roommates from college that are dentists, and after working with optometrists and doctors with their office automation, I have to say that I was honestly surprised to see that our local hospital, Edwards in Naperville, doesn't have xrays on film anymore. Well... maybe they do, but when you go to get a copy of them, you don't get films, you get a CD. "Hey!" I thought... "technology!".

Then I tried to read them on my Mac only to find out that they are a special file format: DICOM - Digital Imaging and Communications in Medicine. This format is not universally understood by apps like GraphicConverter or the like, but I was able to find that ImageJ could read the uncompressed DICOM files - which is nice because my wife's hand xrays came as uncompressed DICOM, but when I tried to load my son's rib xrays it said that it wasn't able to read compressed DICOM files. Crud.

I did a lot more googling and then came across OsiriX. What an amazing piece of open source software. This even has awards from Apple for it's excellent design - and tutorials on the Apple web site about how to use it. This is better than many large commercial outfits and yet it's totally free.

Let me give you a quick run-down of the application. First, it's Universal, and my guess is that they'll update it to Leopard to get the 64-bit processing on the Intel hardware as I believe it's capable of being 64-bit on G5 hardware. Second, it's got a built-in database that's smart enough to allow you to run it in a 'cache-only' mode (more on this later), but also capable of loading up xray series off CDs and keeping them on disk for later viewing. Thirdly, it's got automatic update checking, 'metal' theme (soon to change with Leopard), and responds very quickly to all user requests. It's a top notch Mac app.

One of the first things that made me say Wow! was the fact that on the toolbar there's an icon for importing a complete CD of xrays:

Local DICOM Database

I hit that, and whoosh! everything from the CD is available in the app! I was stunned. I've been trying to get these xrays viewable for about 3 months, and all I had to do was to get this one program for my Mac, and click a button and everything is there - all the fields populated with when/where/why, etc. That's an amazing piece of coding!

But it didn't end there... oh no...

The default behavior is to have the application remove the entries from the database as soon as the CD is unmounted. This makes sense if a doctor is using it - there's no need to keep the images around if the disk is out of the drive. But for me, I want to keep the images around so I don't have to have the disk with me to look at the xrays. In order to do this you need to go into the Preferences for OsiriX and go to the 'Database' pane and select Copy files to OsiriX Data folder if: and then select If files are on a CD/DVD. This will mean that all the files will be copied from CDs. But we're not through yet... Next, go to the 'CD/DVD' pane and uncheck the option: Automatically remove images from the database when CD/DVD unmounted. This will then make sure that the images remain in the database once you remove the CD.

With this, I can carry around all the xrays of my family's accidents. I know it may sound a bit grizzly to some, but it's one of those things that makes me feel more connected to them.

Added More Legend Flexibility to the BKBaseGraphApplet

Tuesday, November 6th, 2007

comboGraph.png

The developer that has been doing a lot of work with the combo graph, and just graphs in general, came to me again today asking if it were possible to make the legend on a graph a fixed size. I didn't think so, but I didn't know for a fact. I knew that you could place the legend on the top of the graph, left, right (default), or bottom - as well as 'floating' it over the graph. I had put that into the base graph a while back, but failed to put it in the applet.

The problem is that if you are displaying graphs stacked on top of each other in a web page, you run the risk of having the legends different sizes (widths). If so, then the graphs themselves will be different sizes and the x-axes will not line up nicely one on top of the other. So the idea was to be able to set it to a fixed size large enough to hold all the legend entries on all the graphs, and then they would line up nicely.

What I found was that VantagePoint isn't as helpful here as it could be.

If you set the width of the legend you must also tell VantagePoint that you don't want it to sutomatically configure the size of the legend. This leads to the problem that if you don't set the height, then the default value is basically only one entry and it looks very bad. So when we set the width, we also have to set the height. Bummer there, folks. But thankfully, it looks like the scale is approximately 20 pixels for each entry in the legend.

Thankfully, I have the ability to see what's supposed to be on the graph, take that number, multiply it by 20 and use that as the height. It's not perfect, but it's pretty good. There might be problems if you change the font size, or even the font. Or if there is meant to be a scrolling list of legend entries and that's going to mess things up something fierce. But for a first cut, and because I've spent hours trying to figure this out with no luck, let's go with this for now.

If it turns out to be a bad guess, then I'll send an email to VantagePoint and ask them how to do a better job of estimation. I'm sure they have to have something that calculates the default height, all I need to do is to tap into that. May not be easy, but it's certainly an avenue if this doesn't work out.

They Don’t See the Value of Defensive Programming

Tuesday, November 6th, 2007

cubeLifeView.gif

I know I've said it before, but one of the most important things I think a professional developer can do is to program defensively. There's no reason not to - it doesn't take that many CPU cycles to check inputs and return values, and the benefits are almost incalculable when you need them. Case in point was a problem I ran across yesterday afternoon and took until this morning to clear up.

Several years ago, a co-worker put together a nice object representation of the instrument data sitting in the MarketMash server. He created this because the current model we were using, and still use to this day, is very amorphous, and not well defined. Nothing like you'd expect to see if you were a traditional java developer. So he decided to make a new object model that would sit beside the existing model and provide a lot more structure to the data and therefore make it a lot easier for people to get at the data they are interested in.

The problem with a lot of structure is as things evolve, you have to maintain that structure a lot more actively than the amorphous model. This came to pass yesterday as I was looking at the output logs of one of the processes of a system of mine. I had put into CloudCity the ability to handle logging of new instrument-level tags once per instance, and then simply ignoring them after that. What I hadn't done was to do the same for the position-level tags, and that's what got me.

Well... what really got me was optimistic programming. Such might be the case as the example:


    CCToDNAMap  dnaMap = CCToDNAMap.getInstance();
    dnaPositionTag = dnaMap.pairFromTag(tag).getName();

In my problem yesterday, it was the call to pairFromTag() that was returning a null and not getting checked for. The additionally odd problem was that this was not throwing a NullPointerException as you'd probably expect. In fact, it was throwing an Exception, but that exception was null. When I broke out the calls and tested every return value, things started working just fine.

I then took the time to add in the new fields, but the real issue was the optimistic programming. For if I add another position-level attribute now, it'll log it once and then continue to work as if nothing bad happened. That's what I wanted to happen in the first place.

Converting to Latest gfortran from Mac HPC

Monday, November 5th, 2007

xcode.jpg

In preparation for Leopard, I decided to see if the latest version of gfortran from the Mac HPC guys would build my thesis work. As little as about 6 months ago it wouldn't - there were issues with it that were supposedly fixed, but the build wasn't on the latest release. I stuck with g77 because it was Tiger (10.4), and it worked. But with Leopard using gfortran and no g77 port, it seemed like a good idea to get the latest gfortran and give it a try.

Interestingly, when I compiled it it compiled cleanly, but on linking I got the following:

    /usr/bin/ld: warning can't open dynamic library: /libgcc_s.1.dylib referenced
    from: /usr/local/lib/gcc/i386-apple-darwin8.10.1/4.3.0/../../../libgfortran.dylib
    (checking for undefined symbols may be affected) (No such file or directory,
    errno = 2)

I did a little searching and found that the solution was to change the 'install_name' (refer to a previous post about Xcode 2.4+ needing this) from /libgcc_s.1.dylib to something like where it's really located: /usr/local/lib/libgcc_s.1.dylib.

To do this simply go into /usr/local/lib and type:

    sudo install_name_tool -change /libgcc_s.1.dylib /usr/local/lib/libgcc_s.1.dylib
      libgfortran.3.dylib

(all one line). With this, the linking problem disappears and the code runs great. Now I know that I don't need g77 on Leopard. Nice. All we're doing is changing the 'install_name' in the gfortran library to have the complete path name as opposed to the simply leading '/'. Nice little tool to have around when issues like this come up.

Ever Closer to Leopard

Friday, November 2nd, 2007

Today I got two good pieces of news... first, PostgreSQL from Marc Liyanage is running fine on Leopard. Second, he's got PHP running on Leopard even if he doesn't have the package ready to install. It's getting closer. Ever closer.

I figure another week and he'll have it all buttoned up.

Another bit of good news, the Mac HPC guys have gfortran for Leopard going. They probably aren't going to get g77 going on Leopard, but so long as my code works on gfortran, I'm a happy camper.

Yup... I'd guess another week and I can get Leopard.

UPDATE: Marc's got it basically working as the Apache plugin, but there are a few issues with the command-line usage. So... it's getting closer, but not quite. I've got work this coming Saturday that will keep me from doing it then. The weekend after that, we've got the Philly Marathon, so I can't do it then. So, since I didn't get it this weekend, I'll have to save it for several weeks.

Creating a Hybrid Java Preferences Framework

Friday, November 2nd, 2007

Creating a Better FileSystemPreferences Framework

Thursday, November 1st, 2007

BKit.jpg

I have spent the last few days - on and off, talking to a developer about the need for a global user Preferences system. The one that comes with most JDKs is decent but there's a problem using the one that ships with Windows - it uses the Registry. That's not the issue as much as that sets up the problem. The user preferences part of the Registry can be put in an unreliable state if you log into two machines and aren't very careful about what you're doing.

Imagine you have two Windows boxes: A and B. On these boxes you have an application - it resides on a server so it's up to you where you launch it. You decide to log into A and then B and launch the app on A. You move a few windows, make a few changes - basically change the Java Preferences values. When you exit the app, your changes have been written to the Registry on A. So far so good. If you logged out of B before logging out of A then the Registry will remain correct and the changes from A will be written to the PDC. But what if you log out of A first?

Well... your changes will be written to the PDC, and then when you log out of B you're going to have that Registry's state written to the PDC. You've lost your changes. No way to get them back.

This is a common problem for a lot of our users. They have multiple machines and we'd like to use Preferences as it's easy and clean, but there is this problem. On Linux/Unix this isn't an issue because the user Preferences are stored in your Home directory. Write it there and you're good to go. Login/logout order doesn't matter. But it does with Windows.

So we decided that it would be a good idea to use the FileSystemPreferences system that's used in Linux for Windows. The problem is that it's not available in the Windows JRE because it's not considered necessary. What a mistake. I've spent the day getting the source code for the FileSystemPreferences and FileSystemPreferencesFactory and creating BKit versions of them that are cross-platform.

That last part was the real key. The original source code for the FileSystemPreferences wasn't because it used three native methods that really didn't have to be used, but were used because they didn't take advantage of the NIO additions in JDK 1.4. I cleaned up all the code, got rid of the native methods, removed the silly XML dependency and used the PList work I had just done on the BKHashTree to serialize these maps to files.

In the end, we have something that's a drop-in replacement for Preferences that will work on all platforms. This means that we can point the user Preferences to a shared disk and so all flushes of the Preferences will write to the files and there won't be any problems like we're having now with the Registry. This is going to make things much nicer.