Archive for the ‘Open Source Software’ Category

Making a tarball of CVS Changes

Tuesday, January 15th, 2008

Apple-logo.jpg

One of the things I like to do when I've made changes to several files in a CVS directory - and haven't yet checked them in, is to make a tarball (a .tgz file) of them so that I can move them to another machine for testing. For example, working on cross-platform code, it's nice to get it working on one platform and then move it to the next and verify it before checking it in. This way all the changes for the modification are in the one CVS check-in.

I wanted to make a simple alias for TCSH that did this on my Mac. I was doing it all the time and it just seemed smart to finally do this right once and then just use it from here on out. The problem I got into were these "Apple Dot" files (._file) that tar was including in the tarball even though they weren't listed in the list of files to include - nor could I exclude them with the command options.

So I did some googling and found the COPYFILE_DISABLE environment setting in Leopard. If this is set to 'true', then these Apple Dot files will not be copied. But, since I didn't want to make a big change to all my shells, I decided to simply make it defined for the one command.

Finally, then, here is the alias:

  alias tcu 'setenv COPYFILE_DISABLE true; tar zcvf \!* `cvs -q update |
      grep -v \^\? | sed -e "s/^[AM] //g"`; unsetenv COPYFILE_DISABLE'

(it's all one line, of course)

Using CKStrings in hash_set and hash_map

Thursday, January 10th, 2008

CKit.jpg

Yesterday I was starting to work on my next little project - a price injector from our price feed to a new vendor app that we're integrating.

Another developer is building a nice C++ wrapper around the proprietary messaging API. It's not terribly complicated, but he's going to wrap it so that we can make a message system that we can use within the Shop without having to worry that we're overloading the vendor's product. So with that part done, or at least getting done, I can focus on the application that will takes prices from the price source (CacheStation) and then send it to the app via our message bus.

Configuration for these apps is something I've worried about over and over for each new app. You want something that's easy to edit... something that's got a central location for easy maintenance... yet not introduce more system dependencies than you absolutely have to. Most of the time this is a configuration file, but I have used configuration databases at times and while I like a lot about that, I think the better solution is to have a flat file that can hold complex objects and structures and then make it possible to have this reloaded on-demand or on a timer and in that way you simply edit the file and the changes will be read in.

So in doing this I was thinking I'd mimic the BKHashMap and BKHashTree in BKit. Both have been very successful for me, and so I started looking at the hash_map in the SGI extensions to STL that have recently been adopted by C++. The problem was that in order to use a CKString as a key in the map (or an element in a hash_set) I needed to specialize the hash<T> template. It was pretty simple once I looked at the header for hash_fun.h:

  /*
   * While I'm not a big fan of the way they have chosen to put in the hash_map
   * and hash_set templates - I think it's too clunky. They should have made
   * it possible to put a hash operator or method on the class, but that's an
   * opinion. Anyway, given how they chose to do it, this template specialization
   * allows us to create sets and maps using the hashing function below.
   */
  namespace __gnu_cxx
  {
    template<>
    struct hash<const CKString &amp;>
    {
      size_t operator()(const CKString &amp; arg) const
      {
        return __stl_hash_string(arg.c_str());
      }
    };
  }

With this, I can create the has_map by simply saying:

  #include <ext>
  #include "CKString.h"
 
  __gnu_cxx::hash_map< CKString, int, __gnu_cxx::hash<const CKString &amp;> > map;
 
  map["when"] = 10;

Once I have the hash_map I can build the CKHashMap and CKHashTree classes. These would then form the basis of the file-based preferences system for the application. I've already done the code for BKHashMap to read and write plists of objects, so it'd be easy for me to make nicely structure-rich preferences files and then simply read them in and manipulate them.

The Conflict Between Individual and Group Goals

Thursday, January 10th, 2008

NetNewsWire.jpg

First, this morning NetNewsWire 3.1 was released, with what was announced as primarily a performance-improvement based release. This is great news. Not that NetNewsWire was a pig, or anything, but when they take the time to work on the code code of an app (like I have been doing to the server) it shows a commitment to the app at a level you don't see when you're just trying to add new features to make users happy. So, Great Job to Brent S. on the release. I'm running it already.

The next bit of news that comes from Brent and the NewsGator folks is that NetNewsWire is now freeware. That's right - totally free. All of the NewsGator 'personal' products are free. They look at this as the way to get mind-share and name recognition and it's probably going to work. There are a lot of people using something like NetNewsWire for personal reasons, and if they see a professional app from the same guys, they'll be more likely to give them the nod - all else being equal.

And at the same time, there's a great article on Rogue Amoeba about the devaluation of all software for a platform when a good piece of software is available for free. The posting says:

There's certainly a place for free software. But when a fully-featured product such as NetNewsWire is suddenly free, it effectively reduces the value of other for-pay software products. "Why should I have to pay for quality tools, when this quality tool is free?", the thinking goes. When something is given away for free, its perceived value is lowered. If software is treated as valueless, it becomes much, much harder to sell. One need only look at a quote from news coverage discussing the move to freeware to see this in action: "And thank [NewsGator] for this grand gesture...While you're at it, do pray that a few other software companies get inspired by this move and follow suit."

So very true. He continues with the browser battle and how when Microsoft started, it was far far behind Netscape, but they threw their weight (and money) behind IE and it became the best choice - and eventually the standard. Then Netscape withered and died. The competition stopped making IE better, it languished, became a petty dictator of the net, and we have what we have today. Power corrupts, and absolute power corrupts absolutely.

Yet at the same time we see that Brent is happy about this decision. Saying in his post:

But I will say that, for me personally, this is a dream come true. Every developer wants to be able to work on the software they love, make a living at it, and give it to the world for free.

Usually you get to pick two out of three — if you’re lucky. Me, I get all three.

So what we have is a situation where the goals of an individual (Brent) collide with the goals of the Mac Development community - possibly even the long-term goals of the Mac Users themselves.

I remember that back in the late 1980s, Prodigy originally test marketed their online service for free - just get the free disk, load it on and get five free accounts. Everything was paid for from the ad revenue. Interesting fact. No one wanted to use it. So they charged $12.50/month for the same thing and they had people signing up by the droves.

Why?

Perceived value. If you don't have to pay for it, can it really be that good? I mean really, if it's free, what's it worth? But if they are asking for money for it, I'll give it a look and see if it's worth that to me.

The decision about NetNewsWire is not that big a deal. Yes, it's going to change the market slightly, and yes, it probably means that RSS readers are going to be driven by NetNewsWire for as long as it's available. But the same it true for TCP/IP stacks. You used to have to pay extra for them on PCs... then they became free downloads... then they were part of the OS... then they were configured automatically out of the box.

The point is, yes, things change. Industries are born, grow, age (hopefully gracefully) and then die. It's life. No matter if it's people, animals or software. And life goes on. Always has... always will.

Adium is 1.2

Monday, January 7th, 2008

Adium.jpg

I can't think of a better poster child for great development on the Mac than Adium. I've been using it ever since I got an Intel Mac as my old multi-protocol chat client, Fire, didn't work properly on the Intel platform. As I was looking for a solution, I noticed that Adium was looking to be the preferred solution for most Mac folks.

I downloaded it, got all my accounts into it, fiddled with the preferences to get it looking and working like I wanted, and have been happily using it ever since.

Today they released 1.2, and the release notes indicate a ton of bug fixes and localization changes. Not that I had that many problems with it before, but it's nice that they are continuing to work on it. If you need something like this, get it - you won't be sorry.

Firefox 3.0 Beta 2 Runs as a 32-bit App on x86_64 Linux

Thursday, December 20th, 2007

Firefox.jpg

This is great news! I wanted to try the latest beta of Firefox 3 - Beta 2 as I've been reading pretty nice things about it. So I go to get the source to compile it on my Fedora Core 5 x86_64 box and realize that I'm missing cairo-gtk, and don't feel like tracking it down right now. The reason I built 2.0.x from scratch was that the tarball of firefox 2.0.x for linux simply didn't run on my x86_64 box in 32-bit more. Simply would not work. I looked at a lot of things to try and get it to work, but no luck.

So I think Hey, I'll give it another try! and downloaded the linux 32-bit code for Firefox 3.0b2. To my complete amazement it worked! I was stunned. I then was able to link in the 32-bit Java plugin just as nice as you please, and Bingo! I'm running Firefox distribution on my 64-bit machine. This is really great news.

I'm glad they have done this. It makes it a ton easier for me to stay up to date with the fixes, etc.

OsiriX 3.0 is Out

Tuesday, December 18th, 2007

OsiriX.jpg

A while back, I got some xray CDs from the hospital for Joseph and Liza and wanted to look at the images because I thought at least Joseph would find them interesting. My travels lead me to locate OsiriX - an incredible Open Source viewer of the DICOM images. Really quite amazing. Well... today they have released ver 3.0 of OsiriX for Leopard (10.5). It's 64-bit, got lots of new features, and in general looks again like it's an amazing feat of Open Source coding.

Now, I'll admit that when I start it I still get the indication "32-bit" in the About box, but again, I'm guessing that it's likely that the "64 bits" support is in some of the libraries as it's not really necessary for the GUI. Minor point.

In general, it's great to be looking at the work. It really is inspirational.

The Value of Tuned Code

Thursday, November 29th, 2007

fortran.jpg

One of the things that I keep coming back to every so often is the simulation work I did during my Ph.D. Now that I had upgraded to 10.5.1 on my laptop, I wanted to see about getting the code going in x86_64 mode. I have had it running in 64-bit on PPC, but now that Xcode supports it, and the gfortran I use does as well, I was interested in seeing if it was going to run faster - or slower, in x86_64 mode.

The first thing I noticed was that the LINPACK routines that I had taken and hand-tuned to the problem were not working out well with the 64-bit compiler. I was getting SegFaults, and rather than mess with trying to fix those versions, I thought I'd use the BLAS and LAPACK that are bundled with Mac OS X in the Accelerate Framework. These are supposed to be optimized for the AltiVec (PPC) and SSE3 (Intel) so I was thinking that moving this way was a nice upgrade.

The code changes weren't major - primarily in the data storage going into the functions, so it only took me a few hours to fix all that up and clean up the code with a few #ifdefs to make it compile either with the LINPACK routines I built or with the LAPACK functions that came with the OS. What was major, were the results.

As I had hoped, the 64-bit version of LAPACK was faster than the 32-bit version. However, the surprise was the fact that my 32-bit hand-optimized routines were faster still. If I wanted, I'm guessing that I could update these guys to 64-bit by looking at the use of the data element sizes - that's got to be the cause of this as the logic is fine, and then I might very well have something that's faster still. What a shot in the arm! I had no idea that the modifications I had made were going to be that fast. Good for me.

UPDATE: I went into the code and found that it was a simple matter of how the integers were being passed from FORTRAN to C. By putting a simple typedef in the code:

    /*
     * Because we need to be able to build this for 32-bit and 64-bit
     * versions, I want to be able to typedef the integer here so that
     * the value coming in from the FORTRAN code matches what we will
     * use here. Without this, we'd have a mess on the conversions.
     */
    #if defined(__x86_64__) || defined(__ppc64__)
    typedef int f_int;
    #else
    typedef long int f_int;
    #endif

and replacing the long int with f_int (FORTRAN int), I was able to use the same code for both builds and the errors went away. Nice.

Picking up a WordPress CodeHighlighter Plugin

Wednesday, November 28th, 2007

wordpress.gif

When I put code segments into a post, I've used BBEdit's Copy as XHTML function to get it out of BBEdit and into the post. It makes something that's impossible to edit in the post, but it looks nice, and it previews nice in MarsEdit. So, all-in-all, not a bad solution.

But it's not perfect.

So I decided to look for a WordPress plugin that would do a similar job, but simply, and without a ton of things I had to manage/deal with. What I found seems to be a nice little compromise between what I had and what should work really well for me. It's called WordPress:CodeHighlighterPlugin (obvious, yes) and it appears to do all the languages I can imaging and it's all just PHP code as a plugin to WordPress. Additionally, to use this you simply use the <pre> tag with an additional lang argument to indicate which language to use. You can also optionally specify line numbers to be generated and where these line numbers should start.

While this plugin doesn't allow me to preview the code in the form it will appear, it does allow me to easily edit it. So it's a trade-off. I'm not sure how much I'll use this guy, but it's loaded and it's there, and should I need it, it's nice to know that it's there. I guess I'll try it out the next time I need to put code into a post.

Getting ctags Working in BBEdit 8.7

Monday, November 12th, 2007

BBEdit.jpg

I was messing with Vim's ctags support today on my Mac and then somehow wondered if either SubEthaEdit or BBEdit had support for ctags like Vim had. I dug into it and in fact BBEdit can support ctags, but unfortunately not the ctags that exist with Mac OS X 10.4 - you need the exuberant ctags on sourceforge.net. They talk extensively about it in Chapter 14 of the BBEdit manual. You need to download, build and install the ctags app:

    cd ctags-5.7
    ./configure --prefix=/usr/local
    make
    sudo make install

In order to have it not conflict with the existing ctags I did the following:

    cd /usr/local/bin
    sudo mv ctags ectags

I also changed the name of the man page to get that matching the new command:

    cd /usr/local/man/man.1
    sudo ctags.1 ectags.1

and to make sure this man page gets into the existing MANPATH - if you don't want to add this location to your MANPATH, you can do:

    cd /usr/local/share/man/man1
    sudo ln -s /usr/local/man/man1/ectags.1 .

Then I created a simple alias that allows me to call it simply with the right arguments for the file BBEdit needs:

    alias ectags `ectags  --excmd=number --tag-relative=no  --fields=+a+m+n+S -R'

So that in the Makefiles for BKit and CKit I can add the target:

    CTAGS = ectags  --excmd=number --tag-relative=no  --fields=+a+m+n+S -R
    ...
    tags:
        @ $(CTAGS) `pwd`/src

And this way I can then automatically pick up the tags in both projects. It's nice that CVS doesn't try to update the file called 'tags', and placing it at the top of the source tree allows BBEdit to find it - as well as Vim, if I'm into that as well.

In the end, this is a really nice little addition. I'm a little surprised that the BBEdit folks didn't include it as a part of the BBEdit distribution so that you would not have to download it and build it. I mean really they know that Mac OS X 10.4 doesn't come with it, and they are going to need it, so I can't quite figure out why they didn't include it. But they didn't. Easy enough to download and build.

UPDATE: I also found that there are 'Jump' and 'Jump Back' menu commands that I've got hot keys set up for. This makes it very easy to jump to a functional definition based on the ctags and then back to where I was, and then back to the definition. Very nice. Gotta love BBEdit for this.

Interesting Argument about Boost Smart Pointers

Thursday, November 8th, 2007

cplusplus.jpg

In my work today on updating the valuation library to a current production version, I once again came across Boost smart pointers. Now there's a lot of good things in Boost, and I have barely scratched the surface of what Boost has to offer, but the implementations I've seen that use smart pointers are much more confusing than they are helpful. After all, the point is to make it easier to write code - not harder.

Well, I was chatting with a good friend about this today after the fact and he uses Boost's smart pointers a lot and doesn't see the confusion. I can agree that if I were using them all the time I'd probably be desensitized to them as well. But I'm not yet used to Boost's smart pointers, and honestly prefer to handle the memory management myself - that's one of the reasons I'm using C++ in the first place - careful resource management. But I see his points, and because I found the conversation very interesting, here's the gist of it.

Take the following little code snippet:


    typedef EQS::Shared<Operation>::Ptr OperationPtr;
    typedef EQS::Shared<Results>::Ptr ResultsPtr;
    ...
    OperationPtr    lOperation = lFactory.getOperation(mType);
    ResultsPtr      lResults = lOperation->execute();
    double          value = lResults->getValue();

While it's perfectly legal code, if you separate the typedefs into the header file and the remaining three lines into an implementation file you have something that's confusing to a traditional C/C++ developer. Are lOperation and lResults pointers or not? Well... in reality, they are and they aren't.

They are in the sense that you can use the "->" to access methods, but in another sense, they are removed when they go out of scope in the code - like traditional stack variables. While this might be seen as a benefit by some, to me it makes the code exceptionally confusing.

In my opinion the line is being drawn too finely. Make theme appear much different from pointers and the confusion goes away. For example, the following code looks odd, and different, but there's no possibility that someone will be confused by the pointer-nature of the "->":


    typedef EQS::Shared<Operation>::Ptr OperationPtr;
    typedef EQS::Shared<Results>::Ptr ResultsPtr;
    ...
    OperationPtr    lOperation = lFactory.getOperation(mType);
    ResultsPtr      lResults = lOperation..execute();
    double          value = lResults..getValue();

or, as if taking a page from Objective-C, this:


    OperationPtr    lOperation = lFactory.getOperation(mType);
    ResultsPtr      lResults = lOperation[execute()];
    double          value = lResults[getValue()];

In both cases, there is a distinct visual difference in how the objects are being used. In these examples, this is silly and trivial, but in large code sections where there is a lot of processing and the usage of the "->" implies traditional pointer, it's confusing.

I know they did this to make them seem as close to real pointers as possible and allow for the auto-CG, but I think they'd be better off - as C++ was with references, in making a new language element. And if they can't have that, then at least overload a different operator - or something to make it looks significantly different from traditional pointers.

I guess my point is that by using smart pointers you'd never think twice about doing this:


    SmartPtr    p = foo.bar();
    p->goof();

but you'd never do this:


    char        p = foo.bar();
    p->goof();

and while you could do this:


    typedef char *CharPtr;
    ...
    CharPtr     p = foo.bar();
    p->goof();

why would you?

Consistency... that's an important part of a language to me. Perl is nice, and yet it's strength is also it's weakness - you can do the same thing a million different ways. But almost all my Perl code does the same thing the same way - just for that consistency.

The conversation has continued and my friend has pointed out that there is a significant historical component to this that I wasn't aware of, and it sheds light on why they are used this way:

fair enough, call it something else, but, history helps shed light here. In the early days of c++, you didn't have any such thing. Then sometime after the first standard release, they said, hey, let's address a simple memory leak issue and added auto_ptr to the stl. Now auto_ptr seems like a reasonable name and they designed it so that you had almost no coding impact to replace your regular c-style pointers with auto_ptr. You could imagine the resistence they would get if you had to re-write huge chunks of code to migrate over to it. Instead, you had to change your type declarations and remove some deletes and that was basically it.

However, people started to realize that std::auto_ptr wasn't so good for various reasons and the boost guys came along and introduced a richer set of pointer objects that kept the same semantics so that it was easy to migrate.

I can see his point, but when he pressed me about what I thought C++ should do I said that they needed to add a fourth elemental data type:

  • value
  • pointer
  • reference
  • smart pointer

and that smart pointer is going to have a different decorator for accessing it's methods and ivars. By making it too close to a pointer, they have in effect made it more confusing.

Where Java Got it Right

After thinking about this overnight, I have to admit that this is where Java got it right. If they had wanted to add in a reference-counted dynamic object in C++, they should have created smart references. Had they done that, it would have allowed for the 'new' without the need for a 'delete'... it would have used the '.' as the method invocator, and it would have allowed for the complete absence of the '*' in definitions. In fact, I suppose that the only thing they are missing now is the use of '->' when '.' would be better. Then, simply call them smart references and the confusion is gone.

You can still have them hold a NULL, unlike traditional C++ references, but that would be the edge condition, as it is in Java, and not the typical use of the datatype. Yup, I have to say that they'd be so much better off in my mind if they went for 'references' as opposed to 'pointers'.