What's it all about, Alfie?

Yesterday I was putting in some new hardware for an app that I have that's been running nicely for a while. Unfortunately, the machine is old, getting underpowered for the task at hand, and it was time to update the hardware. At the same time, I wanted to update the message queue system that it was using. About once a month (or so) the existing message broker (Sun ONE Message Queue 3.0.1) will get into a confused and I'll have to restart it. This is bad as it's in a place in the infrastructure where it means that I'll have to restart several things because of the loss of the connection to the message broker.

Now that Sun has open-sourced the message broker, calling it OpenMQ, I decided to update to it and see if it was any better. I got the latest OpenMQ binaries (ver. 4.1) and then unpacked them using jar. There were precious little in the way of instructions, and in the previous version I had used (3.0.1) no configuration was needed. But that would prove to be a mistake for me.

The proper was to configure OpenMQ 4.1 given that it's installed into a directory called $INST is to first update the $INST/mq/etc/imqenv.conf file to include the proper value for IMQ_DEFAULT_JAVAHOME to the location of JDK 1.5.0. For me this was simply:

    IMQ_DEFAULT_JAVAHOME=/usr/local/jdk1.5.0

then I wanted to have the default memory size set to at least 4GB, so in $INST/mq/bin/imqbroker look to the line around line 82 that looks like:

    _def_jvm_args="-Xms32m -Xmx128m -Xss128k"

and change it to what you want. In my case, I changed it to:

    _def_jvm_args="-Xms32m -Xmx4096m -Xss128k"

since I was running a 64-bit JDK 1.5.0 this was going to work out nicely.

The next thing I needed to do (but didn't do at the time) was to set the maximum individual message size to unlimited. The default is 70MB, but there are some messages in my system that are very large. The place to do this is in $INST/mq/lib/props/broker/default.properties on the line that starts with:

    imq.message.max_size=70m

and change it to what you want it to be. A value of -1 means unlimited, so I set:

    imq.message.max_size=-1

next, for those topics (queues) that are auto-created, there are a few parameters in the section on 'destination based topics' that you might want to use:

    imq.autocreate.destination.maxTotalMsgBytes=-1
    imq.autocreate.destination.maxNumProducers=500
    imq.autocreate.destination.maxBytesPerMsg=-1

the first and last are basically saying "make no limits on the size of an individual message, or the total size of all messages". The middle one is saying that you might want to have a queue that has a lot of producers (injectors) and one consumer (reader), and the default is 100 and it's OK for most folks, but I wanted to make sure that we didn't run into problems if we had clients directly hitting the broker. Now we should be all done with the configuration.

There is an /etc/init.d/imq script in the $INST/mq/etc/init.d directory, but I didn't use that one as I already had one that was working from the 3.0.1 version. All I needed to do was to change the installation location of OpenMQ (the $INST directory) and then it was ready to go. It was already using JDK 1.5.0 and on this machine that means 64-bit.

Add it to the chkconfig start-up on levels 3, 4, and 5 and then start it with /etc/init.d/imq start and all should be just fine.

But that's not how it really happened for me. Here's the problems I ran into to come up with what should have been done in the first place.

First, if you miss the definition of the IMQ_DEFAULT_JAVAHOME variable it means that you have to edit the imqbrokerd script. This was not unusual as I was already in there for the maximum size, so I added that in there and didn't think a thing of it.

Next, if you don't remember to set the maximum message size, then changing it in the default.properties file is not going to be good enough. You have to remove all the data for the instance as I'm convinced that the configuration of the queues is stored in that persistence and the default.properties file is only read when there's nothing in the instance to base the queues off of. So, if you start it without the maximum message length what you need it to be, then you're going to have to shut down IMQ, then remove the entire directory $INST/mq/var/instances/imqbroker. If you don't remove the imqbroker directory I don't think the changes are going to properly work. Thankfully, all my code makes the queues automatically so wiping out the existing configuration is no big deal.

Also for auto-creating queues, you have to remember the two critical size configuration parameters or else you're going to have those queues in trouble when they try to send through large messages. Sure, the 100 producer limit on auto-created topics is reasonable for most installations, but I had talked to another developer here using OpenMQ 4.0 and he had to set it to 500, so I figured that while I was in the config file, I'd up that limit too.

Also, it's important that your app uses the $INST/mq/lib/imq.jar and $INST/mq/lib/jms.jar or else you can have connection problems. Specifically, if going from a 3.x imq.jar to a 4.x IMQ, you're going to get connection errors if you're not using the 4.x imq.jar. So just be safe, get the one with the OpenMQ distribution you're using.

Even since blender was a commercial product, with a free version for SGI machines, I've played with blender. I say played because while I wish I were a graphic artist, I'm not. Not even close.

Oh, I can appreciate good work, and I can do a decent job of things when I have to, but I'm no real artist. I know I haven't really put forth the effort to be a good artist - hours and hours of study and practice, but it's as much the inspiration and native ability that I'm lacking. I'm not a doodler, and that is probably a key element in what makes a person put the effort into the study. So I guess I'm a graphics hacker and leave it at that.

Still... a new version of blender is out, and it runs well on Mac OS X, so I picked up a copy, and should that inspiration finally strike, I know I'll be ready for it.

Yesterday I noticed that the posts to this WordPress blog were an hour off. Very odd. So I looked into the MarsEdit preferences, and then started looking at the WordPress settings as I remembered when the default time of UTC+0 was being first used. What I found was that MarsEdit was OK, and it was WordPress and it's inability for handle Daylight Savings Time that was the real problem. So I did a little digging.

Seems I'm not the first to see this, and they still haven't fixed it. Odd... but the good news was that there was a plugin for WordPress that fixed the DST problem, but it only really worked by setting the TZ environment variable and that meant Unix boxes. No big deal to me, as the host is a linux box, so I picked it up and deposited it in the WordPress plugin directory and then activated it.

Bingo! The timezone could be set to America/Chicago and things seemed to work perfectly. Nicely done. I can appreciate many of the comments on the WordPress web sites that this is not a complete fix for the problem, and I have to agree. It's a hack. A nice hack, but it isn't as universally useful as WordPress itself, and for that, there should be a better fix. But for now... for me... this fixes the problem.

Whew! I've gotten a few very difficult memory issues taken care of in an app I'm working on. Interestingly enough, the points boil down to a few things:

• don't use mutexs in destructors if possible - this one got me on a few occasions
• delete pointers in maps obviously - and don't try to set the value part of the map to NULL after the delete. Use a while loop on the map's empty() method to get all the front() elements in the map.

Before I made sure of these few things I had lots of unusual and hard to pin down memory problems. Now that I've implemented these in all the classes of my app, things are running much smoother now.

You'd think that the following is a valid way to build a destructor:

    std::map<int, char*>	mMap;

    Egg::~Egg()
    {
        std::map<int, char*>::iterator     i;
        for (i = mMap.begin(); i != mMap.end(); ++i) {
            if (i->second != NULL) {
                delete i->second;
                i->second = NULL;
            }
        }
        mMap.clear();
    }

Oh, but you'd be wrong. The problem seems to be in the setting of the NULL into the value part of the map after the (char*) has been deleted. The way to get this to properly run in GCC 3.3.2 on Solaris 8 is to frame the destructor a little differently:

    std::map<int, char*>	mMap;

    Egg::~Egg()
    {
        std::map<int, char*>::iterator     i;
        while (!mMap.empty()) {
            i = mMap.front();
            if (i->second != NULL) {
                delete i->second;
            }
            mMap.erase(i);
        }
    }

Guess there's a good way and a not so good way to do STL things in GCC.

OK, clearly I spent a lot of time trying to get HippoDraw working on my Powerbook. It wasn't obvious nor easy, but there was a certain satisfaction up to the point of hacking at the Python includes. Yet when I get it all complied - and it did all compile, it didn't run for spit. Now I don't think of myself as a pig, or extremist, but when I spend a lot of time building something that's supposed to work - and has been working for a long, long, time, I get a little disappointed.

I'm sorry... I can't believe that they guys that say they got this working on Mac OS X really have. At least not with the versions of the libraries that I have. Of course, the mailing list is down, so I can't find anything about what I might do differently to get this darn thing to work.

So, I trash it. Bummer, but Qt is running fine so it's not that, and the Boost and Python seem to be OK but with no Python experience I'm really not able to debug what's going on there. It's a bummer that Apple broke the NXHost-ing as well otherwise I'd use the NeXTSTEP box for HippoDraw - the one that works!

Well... maybe in a few versions I'll try it again.

OK, I've loved HippoDraw on NeXTSTEP as a wonderful plotting package. It was Open Source and that's great. I believe that CERN did a lot of work on it - they still may, I'm not positive. Anyway, I wanted something for my Powerbook and so I decided to go through the troubles of building HippoDraw for Mac OS X.

1. First, get the sources for Boost-Jam. This is a build tool that the Boost folks have put together. Interesting aside on this point - the Boost folks are putting together what they believe to be the missing object library for C++. Interesting that they went the library route when the STL group (obviously) took the templates route. But having read the docs for HippoDraw, it's clear that I'm going to need to get the Boost.Python library going, so I'm going to need Boost.

   So... go to the Boost web site and pick up the sources for boost-jam as well as Boost itself. There are pre- built binaries for Linux, etc. but nothing for Mac OS X. I got boost-jam 3.1.9 and Boost 1.31.

2. Now we need to build boost-jam. After unpacking it into a directory, build it with:

   % ./build.sh darwin
   

   and then put the results of the build into /usr/local/bin with:

   % sudo cp bin.macosxppc/* /usr/local/bin/
   

3. Next, unpack and build Boost for Panther with Python 2.3:

   % cd ~/Developer/boost_1_31_0/
   % setenv PYTHON_ROOT /System/Library/Frameworks/Python.framework/Versions/2.3
   % setenv PYTHON_VERSION 2.3
   % sudo bjam "-sTOOLS=darwin" install
   

   and then to clean up the installation by:

   % cd /usr/local/include/boost-1_31
   % sudo mv boost ..
   % cd ..
   % sudo rm -rf boost-1_31
   % cd /usr/local/lib
   % ln -s libboost_python-1_31.dylib libboost_python.dylib
   

4. Now build Qt/Mac. First, download the source to /usr/local and then rename the download version (3.3.1 in my case) to /usr/local/qt - or link it if you want to keep the version information. Then:

   % configure -thread
   % cd /usr/lib
   % sudo ln -sf /usr/local/qt/lib/libqt-mt.3.3.1.dylib libqt-mt.dylib
   % sudo ln -sf /usr/local/qt/lib/libgui.1.0.0.dylib libgui.dylib
   

   This is really all in the Qt/Mac docs.

5. On Panther, in the Python includes, I've found a little problem that I haven't really dug into as I'm not a major Python fan. In the Python include directory: $PYTHON_ROOT/include/python2.3 there's a file, object.h. It seems that there's a compile time parsing problem with line 343:

   PyObject *name, *slots;
   

   that needs to be changed to:

   PyObject *name, *user_slots;
   

   in order for things to compile correctly. I've tried to find references to 'slots' but to no avail. I'm not worried about it as I'm not sure the impact, but it's something to consider - and certainly back up object.h if you make the change.

6. On Panther, I've found that isnan(x) is not properly but it's easy enough to correct. In HippoDraw 1.5.1 it turns out that you need to add the lines:

   #ifdef __MACH__
   #ifndef isnan
   #define  isnan(x) ((sizeof(x) == sizeof(double)) ? __isnand(x) : 
                     (sizeof(x) == sizeof(float)) ? __isnanf(x) : __isnan(x))
   #endif
   #endif
   

   at the top of minimizers/BFGSFitter.h to get it to compile. Also, the libtool needs to have a modification. At line 2889 the original libtool reads:

   # Add a -L argument.
   newdeplibs="$newdeplibs $a_deplib" ;;
   

   which needs to be changed to:

   # Add *only* a -L argument
   case $a_deplib in
     -L*) newdeplibs="$newdeplibs $a_deplib" ;;
   esac
   

   Also in libtool you need to add the lines:

   -framework | Carbon | QuickTime | System | OpenGL | AGL)
     deplibs="$deplib $deplibs"
     continue
     ;;
   

   around line 1785 right before the line:

   %DEPLIBS%)
   

   After that, it's a standard build with a few arguments:

   % configure --with-Qt-dir=/usr/local/qt --with-Qt-lib=qt-mt 
               --with-boost-includes=/usr/local/include/boost 
               --with-boost-lib=/usr/local/lib 
               --with-python-include=$PYTHON_ROOT/include/python2.3
   % make
   

One of the things that's been bothering me for a while is the inability for me to run scp and sftp to sparky from my iBook, sherman. I could ssh to sparky OK, but when I tried to get one of the others to work I had a lot of problems. Well... last night I decided to give it a real go and upgrade OpenSSH on sparky with the hopes that this would solve the problem. Of course, nothing's that easy.

Since sparky is a SPARC 20, I went to SunFreeware to get the latest packages for Solaris 7, SPARC edition. I really have to tip my hat to these guys... this is a lot of work to put these packages together for all the different combinations of Solaris and CPU, but they do a wonderful job of it.

Anyway, I get the latest packages for OpenSSH, and realize that I'm going to need to get the latest OpenSSL as well. No biggie, I get both and install them with pkgadd. Then I try to run it. Begin the detour to get it really working. You see, the latest OpenSSH/OpenSSL needs to have an entropy server running so after reading the OpenSSH install page on SunFreeware (which is, thankfully, exceptional) I get the three other packages I needed and installed them. Then it was a matter of about a dozen commands to get the entropy generator going and then OpenSSH ran. I was able to ssh into the box, but alas, still no scp or sftp.

So I go to Google and I get a few ideas - like making sure the path to scp is correctly set. It was, of course, but in doing that I noticed that something wasn't quite right. I was getting an environment variable error, so I went into my .login file and fixed that up. Still no luck, but closer, as ssh sparky 'which scp' worked as it should.

The final step came putting my girls to bed for the night: I've had GNUstep running on this box for a while as I've really liked the OPENSTEP object library, and in my .login the GNUstep app to 'make services' is run. This came to be because there was a process id showing up at each call to ssh, scp, and sftp. When I finally put it together it seemed obvious - GNUstep was sending something or blocking something that was OK for ssh but wasn't OK for scp or sftp.

When I commented out the call in my .login everything worked great! Now I can move files to sparky with scp which is not only faster but a lot more secure than ftp.

I'm very impressed by Safari on Mac OS X, and when decided that the KHTML engine that they used was in Konqueror for Linux I decided to give it a try and see if it would replace Phoenix as my browser on Linux. First off, it's always been clear to me that browsers on Linux are going to suffer from a lack of plug-ins and support for the latest Flash, etc. But that's OK... all I really expect out of my Linux browser is to be able to render standard pages and not hassle with Flash or QuickTime, etc. For that, Netscape was a good first attempt but locked up far too often. Mozilla was better, and Phoenix was even better because of it's stripped-down design on the same Mozilla core.

Then Safari gave me really impressive rendering speeds and I decided to try Konqueror - not bad. But I'm running RedHat 7.1 which is not the latest and so I decided to try and upgrade KDE (and therefore Konqueror) to something a bit more current. So I went and got the 7.2 RPMs and updates for KDE and the necessary support packages. Got it all installed and it's not bad at all. The speed is nice - on par with Safari.

I've been a GNOME supporter because of the CORBA infrastructure and I still believe that using CORBA in this way is the way to make the computer more tightly integrated with the network it sits on. If I had to make a new trading system I'd focus on making the system a group of small, inter-working components that all had universal interconnections that flowed through so that systems could be built by simply plugging components together and when necessary stringing together a few different kinds of components to get the desired function. This is nothing new - CORBA has been all about this from the beginning. But making an OS where this level of interconnection is supported at the lowest levels makes it that much easier to build these types of systems.

Anyway... GNOME is nice, and it's OK, but I have to say that KDE is nice as well and while it's not rooted in CORBA, most current systems aren't. While this isn't a real benefit, currently it isn't a serious handicap either. I haven't been asked to write this mythical system, and were I to do it, I'm guessing that one of the restrictions would be that it had to run on Windows and that would leave Linux out of the running right from the jump. But it's important to support The Team, and so I've been running and supporting GNOME since I started using Linux a few years ago.

The latest RedHat (8.0) solves a little of this problem for me as it has a modified version of both that is it's new desktop of choice. Sure, you can get straight GNOME or KDE on 8.0, but the default is strong motivation to leave it just as it is. I haven't used 8.0 yet, and probably won't until we upgrade all the servers at work which isn't planned anytime soon. After all, they work, and upgrading eight servers along with four workstations isn't going to be fast and there should be a compelling reason to do it.

So I'm running Konqueror on top of Ximian GNOME and have to admit that it's pretty nice, and support for the mixture is really nice.

I got into ARPAnet while in college at Purdue. There was no such thing as the web and http: protocol - it was telnet, ftp, and a few new things like gopher, archie, etc. And there was newsgroups. I loved the newsgroups. They have since really been replaced by web logs and news sites, but in my day they were the place to find all your questions answered, you could buy and sell anything and know that it was a decent, honest, person on the other end because they had figured out newsreaders, after all. Yes, back in the day, it was an entirely different place.

Still, I'm not ready to let go of newsgroups - so I have always tried to keep a newsreader up and going on at least one platform I was near. For the time I was working out of my home office it was
RadicalNews on NeXTSTEP which is an incredible newsreader given when it was written and the hardware it runs on. Really impressive. Then I changed jobs and started carrying my Linux notebook and started using
Pan. Not bad at all. Multi-threading meant that this guy could do things that RadicalNews just could not. It could load all the articles on the subscribed newsgroups so that I could read them on the train. This was nice since I didn't have the time to read it in my office anymore.

Then I got my iBook and started carrying it. So I needed a new newsreader on Mac OS X - none of this Classic for me. I looked at
newsreaders.com for all the OS X readers and went through them one by one. I had to be able to run in OS X natively, and needed to be able to do online as well as offline reading. While I don't mind paying for some software, a newsreader is not one of the things I was willing to pay for. It's just a little quirk about all the news reading on terminals with rn and the other Unix readers. I don't mind working on freeware/open source, so if I had to help build I was ready to do that.

After my search and playing with what passed for demos, I settled on Halime as it did all I wanted to do and had a nice icon to boot. I have been using it ever since. But when I downloaded the 1.0b1 version I noticed that it no longer seemed to be reading the articles from my newsserver. I pulled out one of the other readers I'd tried and sure enough, it looks like the server has the articles, so am I crazy or what?

I spent a day and a half recreating the subscriptions from the massive group listing, trying to fiddle with resetting the article numbers, but in the end nothing I did seemed to matter and I was beginning to think that I was crazy.

So I wrote the author and mentioned my problems. He suggested I crank up the debug level and see what it was saying. I did it and BINGO! I saw a ton of error messages - looked like one for each article header it was trying to read. Ah... I could relax. I wasn't going crazy.

Now I'm waiting for a message from the author who's been a really nice guy about other questions/issues I've had in the past. I'm guessing it's something with the newsserver that I'm using. Probably changed the length of some field or something and that's causing the problems. So I hope that soon I'll get an email about an update to fix the problem and I'll be back to reading news.

This morning I found out something that I hadn't expected to see... that I was probably running both Sawfish and Enlightenment as window managers. I know think that my problems a few days ago with the theme selection were related to this. What I first noticed is that GNOME 1.4 wants to use Sawfish, and that I had Enlightenment as the default in GNOME. So I switched to Sawfish as the default and hopefully things will go much better now. There isn't a significant difference in the window managers - from what I see, but the integration with GNOME certainly tips the scale from my old favorite Enlightenment to Sawfish.

I also was reading a lot in the newsgroups and thought I might be able to catch some reasonably intelligent conversation about SGI and their future. But alas, when I went to a few intelligent-sounding rooms I was greated by the same banter that I've heard time and again in chat rooms. It's too bad, too. Because while I have time to do this, and interesting things to talk about, I can't find an interesting person to talk to. Oh well...

While I was messing around with xchat today I thought I might as well get the latest stable release - 1.6.4 as they are working on 1.7.0 (unstable) now. No biggie, but I think I'll try once more to get them to incorporate my perl fix so that I don't have to hassle with putting it in by hand each time I get the code.