What's it all about, Alfie?

Well... I've been doing a little playing with Coda, and while I have to say it's very nice, and it's got a ton of flexibility. I think it's the tool I've been looking for to do the larger scale, faster AJAX building and deployment. You can put the CSS for a web page in the page, which is quite nice, but you can also put it in a general CSS file, and share it among many pages. Nice flexibility.

Of course, with preview is Safari (WebKit), so the JavaScript is working in the preview, and while it's not exactly like Safari because I can't set my default fonts in the preview pane, it's probably a lot more representative of the typical user than I've been thinking my Safari and Firefox sessions are. For example, in Google Chrome, by WordPress blog looks very different - primarily in the font selection. I'm not sure which is right, because Safari, Firefox and Camino are all looking the same and it's only Chrome that's odd. Maybe it's just Chrome.

Anyway, it's probably a decent thing that the preview is very generic, as it'll force the proper use of CSS. That's important for a whole host of reasons, not the least of which is that these days you have to have CSS or the page looks horrible on just about any browser.

So I was thinking that with Coda, I had a really nice tool to build the pages. But then there's the complete object-oriented development tools. The plus there is that I have the Google DataTable in Java, and JavaScript, and that serialization is done very nicely, but in general, I really need to think about building up some JSON serialization tools for things like Maps and Lists in Java. It's not a complete 1:1 mapping, but there are a few things I could write that would make it a lot easier to serialize out some standard Java datatypes to the JavaScript world, and that would make it faster to build pages.

Still... writing the Java code isn't the long pole for AJAX development for me. It's more the initial building of the HTML page based on all the ways of putting it together. All the different tags and structures are more than a little challenging on multiple browsers, but it's something I'm getting used to, and in time, I feel, it's something that will come as second nature. But for now, it's the one thing that would benefit from a builder tool.

In the end, it's still one of the most enjoyable development environments I've coded in in a while. Glad that I have the tools to do it easily.

I got a tweet today from Dropbox saying that they have a new stable client in the 0.7 line: 0.7.82 and it's ready for download. I run it all the time - makes sense to grab it and see how it runs. It certainly is one of the most impressive solutions to a problem I've seen.

This morning I saw that Perian 1.2 is out, and as Wil Shipley tweeted yesterday:

VLC for the Mac is having trouble finding developers. Why? Well, because Perian rocks. Why screw with another client?

As has been in the news recently, VLC for Mac is on hold because of a lack of developers. I've thought about helping out, but in all honesty, the only reason I need it is to use HandBrake, and there's a perfectly acceptable work-around with FairMount.

If VLC can't attract the Mac developers, it looks like the 64-bit version isn't going to happen, and that means that the 64-bit HandBrake will need to use FairMount. Alternatively, I can just get the 32-bit version of HandBrake until they decide to break their reliance on VLC. We'll have to see as things develop.

But until then, Perian is the better plugin for QuickTime.

Adobe recently admitted to a nasty zero-day exploit in their Flash player that's already making the rounds of the nasty web sites. This, combined with my current need for Flash in my work, means that when Adobe announced a Beta of Flash player 10.1, I got it. No reason to be careless about attacks, and might as well get the latest and greatest Flash.

I'm still up in the air about Flash. In general, I don't like it on the web. Everyone uses it far far too much for animations and ads. It's annoying. Also, I've developed a little in it. Not the best platform I've seen, but not the worst. Yet still... the Google Visualization widgets are in Flash, and I really like them. So it's really Google that's saving my belief in Flash.

Today I spent quite a bit of time putting together a new page in my web app. I had a model to work off of, but there were enough new things I needed to work in that I got to stretch a little artistic muscle - thankfully, I didn't pull anything. But it got me thinking how much I'd like to have a layout editor that is sort of like the WebKit Inspector panel - in reverse.

I spent the majority of the time messing with the HTML to get the layout looking the way I wanted. I had to make it look nice in FireFox and WebKit (Google Chrome), which wasn't horrible, but it did take a lot more time than glueing it all together with JavaScript and the servlet backend. So it's got me to thinking a few thoughts.

First, I'm very interested in hitting Coda again and seeing how they handle the CSS. That could make it a lot faster right there. There's no way to do web design on multiple platforms without CSS. I can't imagine it. So... faster CSS means faster development. I'd like to hit Coda harder and see how it handles more complex pages.

Second, I'm interested in the idea that there might be a more accurate preview system. Maybe this is Coda, I'm not exactly sure, but I'm thinking that spending 15 sec. editing JavaScript followed by a minute of redeploying it, refreshing the web browser, checking it, going to the other box, repeating the process, all adds up. If the preview was more like the Interface Builder preview, then I'd know it was wired up and working exactly as I expected.

Third, I can see the great value in the JavaScript Frameworks like DoJo, etc. and now the buzz coming out about Apple's private framework for making iPhone web apps. These all make things nicer, but I'm looking for one to emerge as the standard - at least for the Mac community. There's too much to handle on a low-level to be doing this all over from scratch each time. Cross-browser compatibility among the biggies here. I just haven't taken the plunge - yet.

I guess that's about it for today. It's been a lot of fun because in general, once you get the HTML hammered out (the view), the controller (JavaScript) and model (servlets) are not too bad. Not a bad way to develop an app, really.

I have wondered about this for a little bit... it started when messages from my InBox on my iPhone disappeared, but they were in my GMail web client. Very odd. I filed a few away, and they didn't show up on my iPhone. Odd. So I decided to do a little Googling... turns out, there's a problem with GMail Push.

This thread on the Google Mobile site explains that it's a bug in their system and they are working on it. Unfortunately, I want the Push email, and I'd like to use GMail as it seems to be the only really reliable push email, but now it seems that I can only see an email for 28 days. That's not so great.

The thread was started in Sept 2009, so maybe they'll get around to fixing it. On the downside, it looks like the primary Google employee that was answering questions on this is now not responding to questions. That's not a good sight at all.

Not a good place to be...

UPDATE: Later today, on the way home on the train, I noticed that I was getting the old emails! Maybe the messing with the date mattered, or maybe Google fixed it yesterday. I'll have to keep an eye on it to see if it stays fixed.

Well, today I got a tweet from OmniGraphSketcher saying that 1.1 Final was out and gave a reference to the notice on the Omni web site.

Looks good.

I have been running some tests all morning, and they'll continue throughout the day, but as I'm watching these tests and gathering data, it occurred to me that I wanted to know which of my data injectors was giving me the most data, and by how much. It's not hard to get the counts with SQL:

  SELECT portfolio, COUNT(*) AS hits
    FROM PortfolioData
   WHERE acquired > '2009-12-15'
  GROUP BY portfolio
  ORDER BY hits DESC

and I get a table that looks a lot like this:

...and I've truncated the table because the effect I was looking for is clear - there are a few of the portfolios that are contributing the vast majority of the rows to the table. The problem is, I can't see the real percentage each contributes to the total. And while I've shown the top 10, there are really more than 30, so it's not easy to do the percentage calculation in my head.

I googled a bit and came up with a really simple solution: include the subquery as the divisor in the select statement:

  SELECT portfolio, COUNT(*) AS hits,
         COUNT(*)*100.0/(SELECT COUNT(*)
                           FROM PortfolioData
                          WHERE acquired > '2009-12-15') AS percentage
    FROM PortfolioData
   WHERE acquired > '2009-12-15'
  GROUP BY portfolio
  ORDER BY hits DESC

and to this I get what I was looking for:

With this, I can now see that my top 4 (out of more than 30) contribute more than 54% of the rows in the table. That's significant, and it's nice to know. If I throttle back these four, I have a great deal of control over the total number of rows inserted in a day.

Cool.

I usually don't spend time writing about the funny things I read on the net - face it, there's just too much of it to really write about, but this morning there are a few things from Daring Fireball that are just too funny to not pass on.

The first is from Fake Steve on a conversation he had with the CEO of AT&T. In it he says of the talk about the coming soon data limiting plans from AT&T:

He launches into a mumbling spiel about how Ralph de la Vega didn’t really say what all the papers are saying he said, and he was misquoted, and it was taken out of context, but I’m like, Bitch, please, guys at our level don’t get taken out of context, we write the shit out in advance and we know exactly what we’re saying when we say it and every goddamn word has been vetted and gone over by a team of flacks. So please don’t sit there like a zoo monkey throwing your own feces at me through the bars of your cage, bokay?

I laughed out loud on that one. It goes to say:

I stopped, then. There was nothing on the line. Silence. I said, Randall? He goes, Yeah, I’m here. I said, Does any of that make sense? He says, Yeah, but we’re still not going to do it. See, when you run the numbers what you find is that we’re actually better off running a shitty network than making the investment to build a good one. It’s just numbers, Steve. You can’t charge enough to get a return on the investment.

I hope that eventually, AT&T will come to their senses and make a decent network for the load. If it takes them losing a ton of customers to balance out the demand, or regulation, or something... I'm hopeful that things will turn around for AT&T.

The next funny was about the fake Walt Mossberg on the CrunchPad-cum-JoJo. If even half of what the sock puppet said was true, then it's an interesting turn of events. I'm not one to think it's OK for folks to go back on their words, but I'm a bigger believer in karma, and you reap what you sew.

These both made me laugh a good deal. Just what I needed this morning.

Well, this morning I went to have a look at frosty, the iMac G3 I have in my home office that I use for hosting my CVS repos, Git repos, web services (including SSL and WebDAV). Well... it wasn't good. The drive had died, and rather than mess with getting another 160GB drive and building it up again, I decided to bring a Mac Mini that used to be the kid's computer out of retirement, get a new 500GB drive for it, get a monitor, and install Snow Leopard on it to make it up to date and a little easier to maintain.

What follows is the details of all the little things I had to do to get all the services running on this Mini. It's not too bad, but it's different enough from the old 10.3 install on the iMac that it warrants detailing the differences.

Getting the Base Tools

The first thing I needed to do after installing Snow Leopard on the new 500GB drive was to use the Time Machine drive from my iMac to pull in the basic apps and my accounts to the new box. Interestingly enough, this seemed to pull over the Developer Tools (Xcode, etc.) but upon closer inspection, come critical command-line tools like rlog and rcsdiff, so I had to install Xcode 3.2 from the Snow Leopard disk, and then use Software Updates to get it to Xcode 3.2.1. This got me the command-line CVS tools as well as the RCS tools I needed for CVSweb.

The next thing I needed to do was to update the box with the version of Git from the Google Code project. This was currently at 1.6.5.2. I have since seen that it's at 1.6.5.5, but so it goes... these minor updates come fast and furious. I'm sure in another few weeks, I'll update to the latest version at the Google Code site, but for now, this was the latest, and more than sufficient.

CVS pserver

The first thing I needed to get going was my CVS repository - with the pserver running on the box. This is just so critical to all the code I have it had to be the first thing I got working.

The first step wasn't too bad. I had good backups of the CVSROOT directory - which just happened to be /usr/local/CVSroot in my old server. So I put it back there, and made sure that the CVSROOT environment variable were defined on my account on the new box, and then start to work on getting the pserver going. Since all my old experience had been with xinetd, and I knew that wasn't on Snow Leopard, I needed to create a launchd configuration file for the CVS pserver.

Taking the example I had to PostgreSQL, I came up with the following that I placed in /Library/LaunchDaemons/cvspserver.plist:

  <?xml version="1.0" encoding="UTF-8"?>
  <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN"
    "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  <plist version="1.0">
  <dict>
      <key>Disabled</key>
      <false/>
      <key>GroupName</key>
      <string>wheel</string>
      <key>InitGroups</key>
      <true/>
      <key>Label</key>
      <string>com.apple.cvspserver</string>
      <key>UserName</key>
      <string>root</string>
      <key>Program</key>
      <string>/usr/bin/cvs</string>
      <key>ProgramArguments</key>
      <array>
          <string>/usr/bin/cvs</string>
          <string>-f</string>
          <string>--allow-root=/usr/local/CVSroot</string>
          <string>pserver</string>
      </array>
      <key>Sockets</key>
      <dict>
          <key>Listeners</key>
          <dict>
              <key>SockPassive</key>
              <true/>
              <key>SockServiceName</key>
              <string>cvspserver</string>
              <key>SockType</key>
              <string>SOCK_STREAM</string>
          </dict>
      </dict>
      <key>inetdCompatibility</key>
      <dict>
          <key>Wait</key>
          <false/>
      </dict>
  </dict>
  </plist>

Of course, the /usr/local/CVSroot is supposed to be the location of the CVS repository, and it just happens to be mine, but if you have something different, use it. I've seen examples where folks create a 'CVS' user, and then use the directory /Users/cvs/CVSROOT, or something like that, so it's all in a "user", which does make it easier to move things around.

For me, it's Old School - the old BSD/Solaris/linux background that makes me think in terms of services and not necessarily users. But to each his own.

Once this is done, load it up with:

  sudo launchctl load /Library/LaunchDaemons/cvspserver.plist

and then you should be good to go.

CVSweb

The key to getting CVSweb working is to make sure that the cvsweb.cgi file is correctly configured. Therein lies a tale.

The original CVSweb I had was for NT. There was a great CVS pserver for NT that I used for a very long time. It was wonderful. It allowed me to use a 180MHz Pentium II with 144MB of RAM and three disk drives totaling less than 30GB to be a CVS pserver, a Sybase Database server, an Apache server, and a few other little things. It was really impressive. So I had the CVSweb configured for NT.

Then I moved it to the iMac G3, and had to convert it to Mac OS X. No big deal, just a bunch of trial and error. Finding odd paths, improper commands, etc. and fixing them one by one. Not really hard, but not trivial, either. The problem was I didn't save it! So I had to do it all again. Thankfully, this time, /Library/WebServer/CGI-Executables/ is backed up with TimeMachine so I won't loose it again.

There's a configuration file that needed to be put in a logical place for the CGI script to pick it up. I mistakenly put it in /etc/apache2/other/ directory as the default name - cvsweb.conf. The problem is that the main httpd.conf has an include of all the *.conf files in other/ and as a consequence, it was trying to read the CVSweb configuration file as an Apache2 configuration file. That was an interesting development. In the end, I placed the cvsweb.conf file in /etc/apache2 directory with all the other high-level config files.

In the end, it was a simple CGI file and a few images to throw in the /Library/WebServer/Documents/Images/ directory. Not bad, just took a little time.

Git Using Gitosis

The next thing I wanted to get going was Git service for my Git repos. This needed to support the two methods of access: the git@git.themanfromspud.com method and the web service (git://) method. The first was basic configuration with a git account, which I had as a backup from the iMac G3. So I just had to make the account on the new machine, un-tar the backup from the external drive, and then it's ready to go. Well... almost.

The PATH for the git user needed to include the path to all the git executables. For the Google Groups install, that's /usr/local/git/bin/, and once I added that, things were a lot better.

Made sure the SSH service is turned 'on' from within System Preferences and I could:

  git clone git@git.themanfromspud.com:project.git

To get the git-daemon going, I needed to make another launchd config file, and this one I called /Library/LaunchDaemons/git.plist and it contained:

  <?xml version="1.0" encoding="UTF-8"?>
  <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN"
    "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  <plist version="1.0">
  <dict>
      <key>Disabled</key>
      <false/>
      <key>Label</key>
      <string>com.apple.git</string>
      <key>UserName</key>
      <string>git</string>
      <key>GroupName</key>
      <string>_www</string>
      <key>Program</key>
      <string>/usr/local/git/bin/git</string>
      <key>ProgramArguments</key>
      <array>
          <string>/usr/local/git/bin/git</string>
          <string>daemon</string>
          <string>--base-path=/Users/git/repositories/</string>
          <string>--export-all</string>
          <string>--inetd</string>
      </array>
      <key>Sockets</key>
      <dict>
          <key>Listeners</key>
          <dict>
              <key>SockPassive</key>
              <true/>
              <key>SockServiceName</key>
              <string>git</string>
              <key>SockType</key>
              <string>SOCK_STREAM</string>
          </dict>
      </dict>
      <key>inetdCompatibility</key>
      <dict>
          <key>Wait</key>
          <false/>
      </dict>
  </dict>
  </plist>

and once it's loaded with:

  sudo launchctl load /Library/LaunchDaemons/git.plist

it's ready to go. You can then clone a repo with:

  git clone git://git.themanfromspud.com/project.git

GitWeb

GitWeb is a little different in that it's in the Git source package. In order to get it, I ended up pulling down the 1.6.5.2 source tarball from the source:

  curl -o git.tar.gz http://www.kernel.org/pub/software/scm/git/git-1.6.5.2.tar.gz
  cd git-1.6.5.2
  ./configure
  make

there's no reason to install it, but it's nice to get it made, just to be sure things are working OK on the server. Then, in the git-1.6.5.2 directory, there's a gitweb directory that has the information you need to install gitweb. There's a CGI file, a Perl file, a couple of PNG images, and a CSS file. Get it all configured and installed, and it's not too bad. Just took a little time.

SSL for Apache2

Getting SSL working on Apache2 was something I wanted, and badly, but it wasn't critical. I had everything I needed, and now I was onto the icing on the cake. SSL isn't critical, but it's something I've used in the past for building other sites, so I wanted to have it on this rebuilt server. Thankfully, this article about getting it going on Leopard (10.5) is still accurate enough to get me home.

I crated a 100 year certificate, sure, it's not signed by anyone other than me, but that's good enough for me. It's all working and that was a nice load off my mind.

PostgreSQL Support for Apple's PHP and Apache2

I decided a while back to stop using Marc's PHP builds - they're just too infrequent, and Apple ships a good PHP build, it just doesn't have PostgreSQL support in it. Thankfully, I had this solved already for my Intel iMac, and there was very little I needed to do to get this working. Having all these detailed instructions in the blog is a really nice thing.

PostgreSQL

When I moved off Marc's PostgreSQL builds, the one that I found that was the most successful for me was the KyngChaos wiki build. This has the 64-bit and 32-bit versions in the same binaries, and that's great for the libraries as well as the database engine itself. I decided that it'd be worth it to get PostgreSQL working on this guy - even if I didn't do a lot of heavy lifting with it, it'd be nice to have. Hey... if I need a bigger machine at a later date, I'll just get a Mac Pro and be done with it.

The launchd config file I used was placed in /Library/LaunchDaemons/org.postgresql.postgres.plist, and contains:

  <?xml version="1.0" encoding="UTF-8"?>
  <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
    "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  <plist version="1.0">
  <dict>
      <key>Label</key>
          <string>PostgreSQL</string>
      <key>UserName</key>
          <string>postgres</string>
      <key>RunAtLoad</key>
          <true/>
      <key>EnvironmentVariables</key>
      <dict>
          <key>PGDATA</key>
              <string>/usr/local/pgsql/data</string>
      </dict>
      <key>ProgramArguments</key>
      <array>
          <string>/usr/local/pgsql/bin/postgres</string>
          <string>-e</string>
          <string>-i</string>
      </array>
      <key>StandardOutPath</key>
          <string>/usr/local/pgsql/var/logfile</string>
      <key>StandardErrorPath</key>
          <string>/usr/local/pgsql/var/logfile</string>
      <key>ServiceDescription</key>
          <string>PostgreSQL Server</string>
  </dict>
  </plist>

and again, once it's loaded with:

  sudo launchctl load /Library/LaunchDaemons/git.plist

it's ready to go. You can check the databases with:

  psql -l

WebDAV with SSL

The last thing I wanted to get going was WebDAV with SSL. I found several articles about WebDAV on Snow Leopard as it's pretty much built-in. All we need to do is to configure it. The one I used as my primary reference was this guy, and it's close - but not exact. The problem is that the config is using Basic authentication and I wanted to use Digest, as the Apple file suggests. In general, I copied a lot more of Apple's example config than I did the article, but it helped put a lot of it in perspective.

The /etc/apache/extra/httpd-dav.conf I got working was:

  DavLockDB "/Library/WebServer/WebDAV/DavLock"
 
  Alias /webdav /usr/local/davroot
  DavMinTimeout 600
  <Directory /usr/local/davroot>
      Dav On
 
      Order Allow,Deny
      Allow from all
 
      AuthType Digest
      AuthName frosty
 
      AuthUserFile "/Library/WebServer/WebDAV.passwd"
      AuthDigestProvider file
 
      <LimitExcept GET HEAD OPTIONS>
          require user drbob
      </limitExcept>
      <Limit GET HEAD OPTIONS>
          require valid-user
      </Limit>
  </Directory>

and then created the necessary password file as stated in the article. In the end, Cyberduck didn't work because it only understands Basic authentication. Transmit understands the Digest authentication, and so that's working just fine. I'm going to see if there's something I can do to get Cyberduck working, but even if I don't it's better to be hitting it with non-plaintext passwords than passing any passwords in the clear. Ever.

All Said and Done

When it all was said and done, including a multi-hour initial TimeMachine backup, I had a new server that had everything I needed and didn't loose a thing from the dead machine. Nice.