What's it all about, Alfie?

This morning I noticed that the Mozilla group has finally released Firefox 3.6 - with a stunning list of changes. Among the biggies for me:

• Protection from out-of-date plugins to keep users safer as they browse.
• Improved JavaScript performance, overall browser responsiveness, and startup time.
• Support for new CSS attributes such as gradients, background sizing, and pointer events.
• Support for new DOM and HTML5 specifications including the Drag & Drop API and the File API, which allow for more interactive web pages.

but there's a lot more there for everyone.

When there's a major update to Firefox, like this, all the themes and plugins fail until the developers have a chance to update them to the new system. This morning was no exception for me. I had to mess with quite a few things in order to get Firefox working on Mac, Windows, and Linux.

Theme Problems - Whitehart 3.6.5 [XP & Linux]

I use the Whitehart theme for XP and Linux and of course it's not ready. Sadly, there's nothing I can do here as it's not in my skill-set to mess with the theme. I'll just have to live with the default theme until they get around to updating it.

XML Rendering Problems [XP & Linux]

I need to look at the XML output of several different servlets and because the servlet's content-type is text/html as opposed to application/xhtml+xml, I don't really see the XML - just some of the unbounded data in the XML. This isn't very useful. The old plug-in I used to force a content-type switch wasn't compatible with Firefox 3.6, so I tried Force Content-Type 1.2.2. What a gem!

I simply install this, activate it, and then put in the mapping for the URLs I need using JavaScript regex and BINGO! It works like a champ! I spent nearly half a day initially trying to figure out what the problem was, but this time, this guy had me up and running in less than 15 mins. That's fast.

If you need to see XML in Firefox on XP or Linux get it.

Java Plug-in [Linux]

This was a puzzler until I started googling it... the Java plug-in for Linux for Firefox 3.6 changed from the old libjavaplugin_oji.so to the new libnpjp2.so. For the JDKs distributed from Sun, this file is in the same basic directory structure, and a simple find will find it. But there's no errors in using the old one, it just doesn't load. Switch to the new one and restart Firefox, and all is well.

This little tidbit wasn't in any of the docs I read, and I wasn't the only one that thought this was an oversight. I can't imagine that after years of using the old plugin, I'd have thought to switch on my own. I'm sure glad one of the developers spoke up on this thread. Yikes!

While doing some network testing/optimization recently, one of the network guys suggested I look at the jTDS parameter packetSize. He thought it might be something to look at if all else failed.

Since I had pretty much gotten to that point, I decided this morning to do those tests, and at the same time take a look at what H2 might say about performance tuning - as that was the destination of the data, after all.

The first step was to change the datatype in the database. According to the H2 docs:

Each data type has different storage and performance characteristics:

• The DECIMAL/NUMERIC type is slower and requires more storage than the REAL and DOUBLE types.
• Text types are slower to read, write, and compare than numeric types and generally require more storage.
• See Large Objects for information on BINARY vs. BLOB and VARCHAR vs. CLOB performance.
• Parsing and formatting takes longer for the TIME, DATE, and TIMESTAMP types than the numeric types.
• SMALLINT/TINYINT/BOOLEAN are not significantly smaller or faster to work with than INTEGER in most modes.

The DBA I'd worked with to set up the back-end database that I read from didn't like using the double datatype primarily due to rounding. I said it was OK, but relented when he pressed. I then used the same DECIMAL(19,6) in the H2 in-memory database as existed in the MS SQL Server database. Seems reasonable, but it flies in the face of the suggestion from the H2 docs.

Since it's all Java, and a Java Double is OK with me, I decided to change all the DECIMAL(19,6) columns in the in-memory database to double. The results were amazing. I was able to achieve more than a 50% increase in the rows/sec processed by this simple change. Additionally, I was able to see a significant reduction in the memory used for the web app after making this change.

All told, a wonderful suggestion.

Then I took to running tests with different values of packetSize. I got:

where the data is (rows/sec) processed from the back-end database into the in-memory database. Faster is clearly better.

What I found was that a size of 8192 was the smallest value that got good performance. So that's what I went with. With these changes, my 7 minute restart is down to about 2:20 - an impressive improvement.

Today I've spent a lot of the day trying to get the restart time of my web app down to a reasonable time. The problem is that I need to load upwards of two million rows from a back-end MS SQL Server back-end database into an H2 in-memory database on restart using the fastest JDBC driver for MS SQL Server I've heard of - jTDS. The reason for this is access speed of the web app. There's just so much data that needs to be available to the servlets that if I were to get it from the database, I'd have an access time that's 10x what I have now.

So I need to load a lot of rows from the back-end database. In the past, I had a restart time that was about a minute. Not horrible. Today I realized that I'm looking at something more like seven minutes. That's too long.

So I pulled in the network guys to see if they could find anything in the wiring in the server room, or settings on the box, because there were machines where the SELECT statements were executed significantly faster than my box. The question was Why?

To their credit, the network guys did an impressive job of digging into the problem. Really amazing analysis of the problem. Unfortunately, in the end, they didn't have anything that was going to significantly change the performance of the query processing for me. But I wasn't too surprised, either. There had to be things I could do to clean things up, and I had always suspected it was going to be up to me in the end.

This morning I was coming off vacation and noticed that Google Chrome for Mac OS X had been updated to 4.0.249.49. Not a major update, but it's nice to see them making progress. Typically, I'm on the 3.x series of Chrome, but that's on Windows, and thankfully I don't have to use that unless it's work-related.

It's great to see Safari get a little competition.

Saw a tweet this morning from Daniel J. that WordPress 2.9 was out. So I headed over to HostMonster and used SimpleScripts to update my installs. It's easy and fast. About the only critique I have is that the backup files are strewn about in my public_html directory and I really have no idea what to do with them. I think I'm going to just delete the old ones and compress the most recent ones. That's got to be good enough for now.

In any case, this little video about the new features is really quite informative, and while I can't imagine doing image editing in WordPress, it looks nice for those that need this capability.

This morning I saw that Perian 1.2 is out, and as Wil Shipley tweeted yesterday:

VLC for the Mac is having trouble finding developers. Why? Well, because Perian rocks. Why screw with another client?

As has been in the news recently, VLC for Mac is on hold because of a lack of developers. I've thought about helping out, but in all honesty, the only reason I need it is to use HandBrake, and there's a perfectly acceptable work-around with FairMount.

If VLC can't attract the Mac developers, it looks like the 64-bit version isn't going to happen, and that means that the 64-bit HandBrake will need to use FairMount. Alternatively, I can just get the 32-bit version of HandBrake until they decide to break their reliance on VLC. We'll have to see as things develop.

But until then, Perian is the better plugin for QuickTime.

Well, this morning I went to have a look at frosty, the iMac G3 I have in my home office that I use for hosting my CVS repos, Git repos, web services (including SSL and WebDAV). Well... it wasn't good. The drive had died, and rather than mess with getting another 160GB drive and building it up again, I decided to bring a Mac Mini that used to be the kid's computer out of retirement, get a new 500GB drive for it, get a monitor, and install Snow Leopard on it to make it up to date and a little easier to maintain.

What follows is the details of all the little things I had to do to get all the services running on this Mini. It's not too bad, but it's different enough from the old 10.3 install on the iMac that it warrants detailing the differences.

Getting the Base Tools

The first thing I needed to do after installing Snow Leopard on the new 500GB drive was to use the Time Machine drive from my iMac to pull in the basic apps and my accounts to the new box. Interestingly enough, this seemed to pull over the Developer Tools (Xcode, etc.) but upon closer inspection, come critical command-line tools like rlog and rcsdiff, so I had to install Xcode 3.2 from the Snow Leopard disk, and then use Software Updates to get it to Xcode 3.2.1. This got me the command-line CVS tools as well as the RCS tools I needed for CVSweb.

The next thing I needed to do was to update the box with the version of Git from the Google Code project. This was currently at 1.6.5.2. I have since seen that it's at 1.6.5.5, but so it goes... these minor updates come fast and furious. I'm sure in another few weeks, I'll update to the latest version at the Google Code site, but for now, this was the latest, and more than sufficient.

CVS pserver

The first thing I needed to get going was my CVS repository - with the pserver running on the box. This is just so critical to all the code I have it had to be the first thing I got working.

The first step wasn't too bad. I had good backups of the CVSROOT directory - which just happened to be /usr/local/CVSroot in my old server. So I put it back there, and made sure that the CVSROOT environment variable were defined on my account on the new box, and then start to work on getting the pserver going. Since all my old experience had been with xinetd, and I knew that wasn't on Snow Leopard, I needed to create a launchd configuration file for the CVS pserver.

Taking the example I had to PostgreSQL, I came up with the following that I placed in /Library/LaunchDaemons/cvspserver.plist:

  <?xml version="1.0" encoding="UTF-8"?>
  <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN"
    "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  <plist version="1.0">
  <dict>
      <key>Disabled</key>
      <false/>
      <key>GroupName</key>
      <string>wheel</string>
      <key>InitGroups</key>
      <true/>
      <key>Label</key>
      <string>com.apple.cvspserver</string>
      <key>UserName</key>
      <string>root</string>
      <key>Program</key>
      <string>/usr/bin/cvs</string>
      <key>ProgramArguments</key>
      <array>
          <string>/usr/bin/cvs</string>
          <string>-f</string>
          <string>--allow-root=/usr/local/CVSroot</string>
          <string>pserver</string>
      </array>
      <key>Sockets</key>
      <dict>
          <key>Listeners</key>
          <dict>
              <key>SockPassive</key>
              <true/>
              <key>SockServiceName</key>
              <string>cvspserver</string>
              <key>SockType</key>
              <string>SOCK_STREAM</string>
          </dict>
      </dict>
      <key>inetdCompatibility</key>
      <dict>
          <key>Wait</key>
          <false/>
      </dict>
  </dict>
  </plist>

Of course, the /usr/local/CVSroot is supposed to be the location of the CVS repository, and it just happens to be mine, but if you have something different, use it. I've seen examples where folks create a 'CVS' user, and then use the directory /Users/cvs/CVSROOT, or something like that, so it's all in a "user", which does make it easier to move things around.

For me, it's Old School - the old BSD/Solaris/linux background that makes me think in terms of services and not necessarily users. But to each his own.

Once this is done, load it up with:

  sudo launchctl load /Library/LaunchDaemons/cvspserver.plist

and then you should be good to go.

CVSweb

The key to getting CVSweb working is to make sure that the cvsweb.cgi file is correctly configured. Therein lies a tale.

The original CVSweb I had was for NT. There was a great CVS pserver for NT that I used for a very long time. It was wonderful. It allowed me to use a 180MHz Pentium II with 144MB of RAM and three disk drives totaling less than 30GB to be a CVS pserver, a Sybase Database server, an Apache server, and a few other little things. It was really impressive. So I had the CVSweb configured for NT.

Then I moved it to the iMac G3, and had to convert it to Mac OS X. No big deal, just a bunch of trial and error. Finding odd paths, improper commands, etc. and fixing them one by one. Not really hard, but not trivial, either. The problem was I didn't save it! So I had to do it all again. Thankfully, this time, /Library/WebServer/CGI-Executables/ is backed up with TimeMachine so I won't loose it again.

There's a configuration file that needed to be put in a logical place for the CGI script to pick it up. I mistakenly put it in /etc/apache2/other/ directory as the default name - cvsweb.conf. The problem is that the main httpd.conf has an include of all the *.conf files in other/ and as a consequence, it was trying to read the CVSweb configuration file as an Apache2 configuration file. That was an interesting development. In the end, I placed the cvsweb.conf file in /etc/apache2 directory with all the other high-level config files.

In the end, it was a simple CGI file and a few images to throw in the /Library/WebServer/Documents/Images/ directory. Not bad, just took a little time.

Git Using Gitosis

The next thing I wanted to get going was Git service for my Git repos. This needed to support the two methods of access: the git@git.themanfromspud.com method and the web service (git://) method. The first was basic configuration with a git account, which I had as a backup from the iMac G3. So I just had to make the account on the new machine, un-tar the backup from the external drive, and then it's ready to go. Well... almost.

The PATH for the git user needed to include the path to all the git executables. For the Google Groups install, that's /usr/local/git/bin/, and once I added that, things were a lot better.

Made sure the SSH service is turned 'on' from within System Preferences and I could:

  git clone git@git.themanfromspud.com:project.git

To get the git-daemon going, I needed to make another launchd config file, and this one I called /Library/LaunchDaemons/git.plist and it contained:

  <?xml version="1.0" encoding="UTF-8"?>
  <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN"
    "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  <plist version="1.0">
  <dict>
      <key>Disabled</key>
      <false/>
      <key>Label</key>
      <string>com.apple.git</string>
      <key>UserName</key>
      <string>git</string>
      <key>GroupName</key>
      <string>_www</string>
      <key>Program</key>
      <string>/usr/local/git/bin/git</string>
      <key>ProgramArguments</key>
      <array>
          <string>/usr/local/git/bin/git</string>
          <string>daemon</string>
          <string>--base-path=/Users/git/repositories/</string>
          <string>--export-all</string>
          <string>--inetd</string>
      </array>
      <key>Sockets</key>
      <dict>
          <key>Listeners</key>
          <dict>
              <key>SockPassive</key>
              <true/>
              <key>SockServiceName</key>
              <string>git</string>
              <key>SockType</key>
              <string>SOCK_STREAM</string>
          </dict>
      </dict>
      <key>inetdCompatibility</key>
      <dict>
          <key>Wait</key>
          <false/>
      </dict>
  </dict>
  </plist>

and once it's loaded with:

  sudo launchctl load /Library/LaunchDaemons/git.plist

it's ready to go. You can then clone a repo with:

  git clone git://git.themanfromspud.com/project.git

GitWeb

GitWeb is a little different in that it's in the Git source package. In order to get it, I ended up pulling down the 1.6.5.2 source tarball from the source:

  curl -o git.tar.gz http://www.kernel.org/pub/software/scm/git/git-1.6.5.2.tar.gz
  cd git-1.6.5.2
  ./configure
  make

there's no reason to install it, but it's nice to get it made, just to be sure things are working OK on the server. Then, in the git-1.6.5.2 directory, there's a gitweb directory that has the information you need to install gitweb. There's a CGI file, a Perl file, a couple of PNG images, and a CSS file. Get it all configured and installed, and it's not too bad. Just took a little time.

SSL for Apache2

Getting SSL working on Apache2 was something I wanted, and badly, but it wasn't critical. I had everything I needed, and now I was onto the icing on the cake. SSL isn't critical, but it's something I've used in the past for building other sites, so I wanted to have it on this rebuilt server. Thankfully, this article about getting it going on Leopard (10.5) is still accurate enough to get me home.

I crated a 100 year certificate, sure, it's not signed by anyone other than me, but that's good enough for me. It's all working and that was a nice load off my mind.

PostgreSQL Support for Apple's PHP and Apache2

I decided a while back to stop using Marc's PHP builds - they're just too infrequent, and Apple ships a good PHP build, it just doesn't have PostgreSQL support in it. Thankfully, I had this solved already for my Intel iMac, and there was very little I needed to do to get this working. Having all these detailed instructions in the blog is a really nice thing.

PostgreSQL

When I moved off Marc's PostgreSQL builds, the one that I found that was the most successful for me was the KyngChaos wiki build. This has the 64-bit and 32-bit versions in the same binaries, and that's great for the libraries as well as the database engine itself. I decided that it'd be worth it to get PostgreSQL working on this guy - even if I didn't do a lot of heavy lifting with it, it'd be nice to have. Hey... if I need a bigger machine at a later date, I'll just get a Mac Pro and be done with it.

The launchd config file I used was placed in /Library/LaunchDaemons/org.postgresql.postgres.plist, and contains:

  <?xml version="1.0" encoding="UTF-8"?>
  <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
    "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  <plist version="1.0">
  <dict>
      <key>Label</key>
          <string>PostgreSQL</string>
      <key>UserName</key>
          <string>postgres</string>
      <key>RunAtLoad</key>
          <true/>
      <key>EnvironmentVariables</key>
      <dict>
          <key>PGDATA</key>
              <string>/usr/local/pgsql/data</string>
      </dict>
      <key>ProgramArguments</key>
      <array>
          <string>/usr/local/pgsql/bin/postgres</string>
          <string>-e</string>
          <string>-i</string>
      </array>
      <key>StandardOutPath</key>
          <string>/usr/local/pgsql/var/logfile</string>
      <key>StandardErrorPath</key>
          <string>/usr/local/pgsql/var/logfile</string>
      <key>ServiceDescription</key>
          <string>PostgreSQL Server</string>
  </dict>
  </plist>

and again, once it's loaded with:

  sudo launchctl load /Library/LaunchDaemons/git.plist

it's ready to go. You can check the databases with:

  psql -l

WebDAV with SSL

The last thing I wanted to get going was WebDAV with SSL. I found several articles about WebDAV on Snow Leopard as it's pretty much built-in. All we need to do is to configure it. The one I used as my primary reference was this guy, and it's close - but not exact. The problem is that the config is using Basic authentication and I wanted to use Digest, as the Apple file suggests. In general, I copied a lot more of Apple's example config than I did the article, but it helped put a lot of it in perspective.

The /etc/apache/extra/httpd-dav.conf I got working was:

  DavLockDB "/Library/WebServer/WebDAV/DavLock"
 
  Alias /webdav /usr/local/davroot
  DavMinTimeout 600
  <Directory /usr/local/davroot>
      Dav On
 
      Order Allow,Deny
      Allow from all
 
      AuthType Digest
      AuthName frosty
 
      AuthUserFile "/Library/WebServer/WebDAV.passwd"
      AuthDigestProvider file
 
      <LimitExcept GET HEAD OPTIONS>
          require user drbob
      </limitExcept>
      <Limit GET HEAD OPTIONS>
          require valid-user
      </Limit>
  </Directory>

and then created the necessary password file as stated in the article. In the end, Cyberduck didn't work because it only understands Basic authentication. Transmit understands the Digest authentication, and so that's working just fine. I'm going to see if there's something I can do to get Cyberduck working, but even if I don't it's better to be hitting it with non-plaintext passwords than passing any passwords in the clear. Ever.

All Said and Done

When it all was said and done, including a multi-hour initial TimeMachine backup, I had a new server that had everything I needed and didn't loose a thing from the dead machine. Nice.

Well... I've sneaked a few developer builds of Google Chrome for Mac OS X, but today I finally got an email from the Google Chrome for Mac team. Now when you go to the Chrome web page it detects that you're on a Mac, and gives you the option to download it for Mac OS X 10.5 or later. It's still 32-bit, but I guess that's not a terrible thing as it's not putting anything into the kernel, which would be a big no-no to getting my laptop to boot 64-bit.

The about page is very nicely done, and it appears to borrow heavily from the Windows versions for the UI, but there are several things that are very Mac-specific, and that's nice to see. Well... I'll be checking it out, but I'll be stunned if it beats Safari. Still, nice to see.

I had to laugh at the email because it listed a "few fun facts" about the Google Chrome for Mac team:

Here are a few fun facts from us on the Google Chrome for Mac team:

• 73,804 lines of Mac-specific code written
• 29 developer builds
• 1,177 Mac-specific bugs fixed
• 12 external committers and bug editors to the Google Chrome for Mac code base, 48 external code contributors
• 64 Mac Minis doing continuous builds and tests
• 8,760 cups of soft drinks and coffee consumed
• 4,380 frosted mini-wheats eaten

it was the last item - the Frosted Mini-Wheats that made me laugh! It's what I have every morning at work. What a gas!

I keep hoping that Miro will get a little more mainstream, but I think it's going to be a lot like Public Television - interesting, and educational, but not necessarily entertaining. Still, it's nice to see them making progress, and there's always time to learn something new on PBS. 🙂

This morning I was playing with the new WordPress 2 iPhone app - putting my blogs into the app, and realized that I didn't have written down the password to my old journal from UBS. This was a mistake, because I remember forgetting it, then changing it and then today I found where I'd written down the coded message that was the password, but forgot the changed one.

Thankfully, the way to change the password is really very simple - assuming that you have access to the databases that drive the system... and I do. So I changed it back to the original value and left it at that. Now I'm ready to play with the new iPhone app, and it works. Nicely, in fact.

While I was at HostMonster, I noticed that WordPress 2.8.6 was out, and so I upgraded my installs to the latest stable release. They had 2.9b1, but I'm not that hyped to use the beta - I can wait.

It's great tools, well put together, and hosted by a great company. Love this set-up.