What's it all about, Alfie?

My daughter is a huge fan of movie-making and her next foray is into the realm of green screen (or blue screen) movies. She's talking about getting a loft bed to make room in her bedroom to put up the screen. That's a clear vision, I have to say.

So I did a little digging and I have to say that there seems to be a lot of plugins for the older versions of iMovie - you take two clips, one that's the background, the other is the screened clip, and then layer the second on the first. It's a nice way of doing things, but as I've also found out, Final Cut Express ($199) does all this and a whole lot more. So that may explain why I haven't seen any plugins that have kept up with the latest iMovie.

Of these, she seems to be most interested in eZedia's eZeScreen plug-in for iMovie 6 (or later). It's a Universal Binary, and says it'll work with anything after iMovie 6, so we'll have to see what happens when we get the demo and try it out.

I've also seen Norrkross Movie, which is sort of an iMovie replacement, and it's not bad looking for $59. It's got all the things she's looking for, and is probably the place to start as it's a nice entry point for what she's doing. If she keeps up with this, Final Cut is a reasonable alternative, but it's after she's kept with it.

I've been working with this software package for months now, and while I can't use the name for fear of lawsuit, I think I'll borrow a page from J.K.Rowling and call it The Software Package Which Must Not Be Named or maybe just You Know What for short. OK, it's silly, but here's another silly lesson to be learned from this million-dollar fiasco: Sometimes, if things are slow enough, they will be technically working, but the users will think them broken.

Case in point with You Know What was a certain instrument's trade ticket. There were supplemental data fields on the form, and they appeared not to populate after being programmatically set on the committal of the trade. I was testing this very thing this morning and after seeing it happen in front of my own eyes, I reached for my pencil to make notes of the fields I'd populated, and the ones that hadn't been correctly populated with the calculate values.

After a few seconds of writing the notes, I looked up to see the empty fields and amazingly, they had been filled. I'm convinced that this was what was happening all along. The way in which this system is wired up is to save to one database, replicate all changes to another and then reach from the replicant. If the changes went in programmatically, then the GUI might not have known to update them, and so they had to go through the replication chain and that was going to take several seconds. It might just be that the users/testers were impatient, and when they didn't see the data in these fields immediately after the saving, they assumed the fields weren't going to be populated and closed the window.

So... this morning I'm going to sit with the tester and make them wait to see if the fields don't appear. I figure that it's about a 30 sec wait, and if they show up for him like they did for me, then we know it's not a bug... it's a feature. Unfortunately, this is a nasty feature, but there's no helping that short of a significant reconfiguration of the system. Don't know if and when they might happen, but if it does, then this delay should lessen. But we'll have to see.

For now, You Know What seems to be OK. You just have to be patient. I wish I felt that described more of the users, but I don't.

Today I was trying to look at the chat interface of one of my apps that uses CKit and has a chat interface. I wanted to make sure that we'd be able to enable logging of individual price updates during the day because some of the support staff in London were getting a little panicked because they're used to this level of logging in another app I've written and frankly, they've come to depend on it.

So I was checking and realized that the chat interface was offline. Odd. It should work. Then it hit me... there was a change I'd made to the chat interface not so long ago (July 23, 2008: Lots of Trouble with MindAlign These Days). But I couldn't remember it. Thank goodness for WordPress! I found the entry where I'd changed the logic on the getReply() method on CKIRCProtocol. I'd had a problem of an infinite loop where I wasn't checking for an error when nothing was returned from the socket. So I changed the code to look like this:

    // now read up to the "\n" NEWLINE that the IRC server sends
    if (!error) {
        retval = mCommPort.readUpToNEWLINE();
        /*
         * It's possible that the data is empty - but the only way for
         * that to be acceptable is for a timeout to have occured. So,
         * if the data is empty and a timeout *didn't* occur, then we
         * need to disconnect this guy and the next pass through, we'll
         * be able to connect again and set things up properly - we
         * hope.
         */
        if (retval.empty() && (errno != ERR_READ_TIMEOUT)) {
            disconnect();
        }
    }

but that left me in the case today where the return code wasn't an error or a timeout - but in interrupt. What to do? Well... the point of an interrupt is that something caused it to bail, but it's not a problem of the socket per-se. So I needed to really change the code to read:

    // now read up to the "\n" NEWLINE that the IRC server sends
    if (!error) {
        retval = mCommPort.readUpToNEWLINE();
        /*
         * It's possible that the data is empty - but the only way for
         * that to be acceptable is for a timeout to have occured. So,
         * if the data is empty and a timeout *didn't* occur, then we
         * need to disconnect this guy and the next pass through, we'll
         * be able to connect again and set things up properly - we
         * hope.
         */
        if (retval.empty() && (errno == ERR_READ_ERROR)) {
            disconnect();
        }
    }

so that I'm sure to only bail when there's an error. Timeouts and interrupts don't count. This fixed up the problem and I got it out to the effect apps. Interesting little thing that turned up on linux. Solaris didn't have this issue.

After I got the WebDAV server working on my OS X 10.4 machine, the next thing was to get SSL working with a self-signed certificate so that I could do secure WebDAV to the box. I've done this on Leopard (10.5) and so doing it on 10.4 was not too bad, but there were a few wrinkles.

First, follow the directions for creating the certificate files in this Mac OS X Hints hint. Don't mess with the httpf.conf file as we're going to do something a little more 10.5-like there. But make the cert, and the private key and the request and then sign it all.

Then in the /etc/httpd/users/ directory use the following file for ssl.conf - theirs is missing a few things.

  #
  # This is the SSL config goodies
  #
  LoadModule ssl_module         libexec/httpd/libssl.so
  AddModule mod_ssl.c

  <IfModule mod_ssl.c>
    Listen 80
    Listen 443
    # Some MIME-types for downloading Certificates and CRLs
    AddType application/x-x509-ca-cert .crt
    AddType application/x-pkcs7-crl .crl

    # inintial Directives for SSL

    # enable SSLv3 but not SSLv2
    SSLProtocol all -SSLv2
    SSLPassPhraseDialog builtin
    SSLSessionCache dbm:/var/run/ssl_scache
    SSLSessionCacheTimeout 300
    SSLMutex file:/var/run/ssl_mutex
    SSLRandomSeed startup builtin
    SSLLog /var/log/httpd/ssl_engine_log            
    SSLLogLevel info

    <VirtualHost _default_:443>
      SSLEngine on
      DocumentRoot "/Library/WebServer/Documents"
      ServerName MACHINE
      ServerAdmin you@yourplace.com
      ErrorLog /var/log/httpd/error_log
      TransferLog /var/log/httpd/access_log

      SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL
      SSLCertificateFile /Users/YOURNAME/Documents/certs/frosty/newcert.pem
      SSLCertificateKeyFile /Users/YOURNAME/Documents/certs/MACHINE/webserver.nopass.key
      SSLCACertificateFile /Users/YOURNAME/Documents/certs/demoCA/cacert.pem
      SSLCARevocationPath /Users/YOURNAME/Documents/certs/demoCA/crl

      <FilesMatch "\.(cgi|shtml|phtml|php)$">
        SSLOptions +StdEnvVars
      </FilesMatch>
      <Directory "/Library/WebServer/CGI-Executables">
        SSLOptions +StdEnvVars
      </Directory>

      # correction for browsers that don't always handle SSL connections well
      SetEnvIf User-Agent ".*MSIE.*" \
        nokeepalive ssl-unclean-shutdown \
        downgrade-1.0 force-response-1.0
    </VirtualHost>
  </IfModule>

Where YOURNAME is the login you put the certificates under, and MACHINE is the name of the machine directory that you put the specific certs in. It's pretty close to what he had, but there are a few differences and it's important differences.

With this, I have https: and WebDAV over SSL for the box. Nothing horribly hard about this, but it's nice to have it all working now. Just took a little time.

With all the interest in using WebDAV as a network storage protocol for things like dotMac and applications like Skitch, I wondered if it would be possible to get WebDAV working on my older Mac server sitting in my office at home. I have heard it's easy if you have 10.5 (Leopard) but I'm not really ready for that, and it was more for the interest of getting it going on the server and not for any real pressing need. If I had a pressing need, then I'd have it going on Leopard, as it's got everything ready to go. This was going to be interesting.

What I found was that there was a lot of people looking into this in the pre-10.5 days, and it wasn't really all that successful in the general sense. Several folks reported they got something working, but they had to disable all the security measures (like encrypted passwords) and that was simply unacceptable to me.

But I got it close working. The WebDAV service was running in the stock Apache from Apple, and I could get it to authenticate with security, and I even saw the files in the root of the WebDAV service. The problem was that I could not do anything with them. I ended up with complete WebDAV functionality.

I could not read them, or update them, or save files... it was like only a limited subset of the functionality I needed was in the WebDAV client that shipped with 10.4.x. And to that I guess I'm not terribly surprised. It would have been really nice, but I'm not surprised that in order to have this really work I'm going to have to be working with Apache2 and the WebDAV module there. [Really, it turned out to be a configuration problem that I really needed to look at the Apache logs to figure out. There was also a configuration problem on the client I was using (Cyberduck) and I needed to figure out the server before I could figure out the client. But I got both.]

But on the off-chance it's possible to get this finished, here's what I did. First, you need to edit the /etc/httpd/httpd.conf and where you see the lines:

  ...
  #LoadModule digest_module          libexec/httpd/mod_digest.so
  ...
  #LoadModule dav_module             libexec/httpd/libdav.so
  ...
  #AddModule mod_digest.c
  ...
  #AddModule mod_dav.c
  ...

edit them to remove the comments so that they become:

  ...
  LoadModule digest_module          libexec/httpd/mod_digest.so
  ...
  LoadModule dav_module             libexec/httpd/libdav.so
  ...
  AddModule mod_digest.c
  ...
  AddModule mod_dav.c
  ...

then, after the block of config that looks like this:

  <IfModule mod_dir.c>
      DirectoryIndex index.html index.htm index.php
  </IfModule>

add in the configuration for the WebDAV server component:

  #
  # This is for WebDAV
  #
  <IfModule mod_dav.c>
      DAVLockDB /Library/WebServer/WebDAV/Dav_lock
      Alias /webdav /usr/local/davroot
      DavMinTimeout 600
      <Directory /usr/local/davroot>
          Dav On
          AllowOverride None
          Options FollowSymLinks Indexes
          AuthType Digest
          AuthDigestFile /Library/WebServer/WebDAV.passwd
          AuthName "frosty"
          <LimitExcept GET HEAD OPTIONS>
              require user drbob
          </LimitExcept>
          <Limit GET HEAD OPTIONS>
              require valid-user
          </Limit>
      </Directory>
  </IfModule>

This ends the editing of the /etc/httpd/httpd.conf file, so you can save it. What will become important later is the location of the directories and files, and the AuthName as that will be used in the password creation command, below.

At this point, it's time to make the location of the 'root' of the WebDAV server. I've chose to hang it off /usr/local, but you can put it anywhere:

  cd /usr/local
  sudo mkdir davroot
  sudo chown -R www:www davroot

and we also need to make a place for the lock file:

  cd /Library/WebServer
  sudo mkdir WebDAV
  sudo touch WebDAV/Dav_lock
  sudo chown -R www:www WebDAV

We're getting close. The next thing to do is to create the password file for the user(s) mentioned in the configuration, above. The AuthName is used here, as is the user drbob as it appears above. To make the password file:

  sudo htdigest -c WebDAV.passwd frosty drbob

Finally, we're ready to restart Apache:

  sudo apachectl graceful

At this time, on Mac OS X 10.4.x (Tiger), the user will be able to login securely to the URL http://machine/webdav/, and they will be able to see the contents of /usr/local/davroot but they will not be able to save things there, nor will they [and] be able to see, modify, or [and] use anything that they see. It's almost like the initial protocol is supported, but the more advanced stuff the clients have updated, but the server has not. [Everything works as advertised.]

Like I said... if it gets serious, I'll put this on my Leopard box and be done with it. But there's nothing pushing me right now, and I may come back to this and try to see what I need to do to fix it. Hard to tell. But that's what I've done.

UPDATE: Holy Cow! I decided to look at the Apache error and access logs and I figured out the problem. The Alias line above needed to not have the trailing slashes and they needed to match. I removed it from the first argument and restarted and all of a sudden, things started working! I'm amazed. Transmit works, Safari works, the Finder works... everything seems to work just fine. I need to set up SSL on frosty, but that should be pretty easy and when that's done I'll have a secure channel to my home WebDAV! How cool is that?!

Cyberduck still tries to assume it's using Basic authentication, and I can see that, it's not trying Digest, but the WebDAV server understands that and deals with it nicely. It's amazing. Wow.

This morning I was checking on a few remote sites and decided to pull up Cyberduck to see how it was doing with the relative speed of access to my home network. When it launches, Cyberduck uses the Sparkle framework to check for updates, and there was a new version released - 3.0.2. Of course, I got it.

Now I have paid for Transmit, and I'll still use it from time to time, but I have to say that I like the minimalistic GUI layout of Cyberduck more. No unnecessary border, no wasted screen real-estate - that's what I like. But with 3.0.2 they've added a default Bookmarks view. All of a sudden, it's not as clean as it used to be.

There's a menu item to toggle the bookmarks on and off - that's nice, but you can't really get rid of the bar right below the toolbar. You used to be able to just have the toolbar and the list of files - very nice and clean. You drag things out and that downloads them, you drop them in and it uploads them. What could be simpler?

I've looked around the program and I can't find a way to remove the second toolbar. Maybe they'll get requests to make it optional - maybe even from me. If they can remove it I'll be a lot happier. I just want a simple interface... nothing fancy or anything I don't need.

After reading several posts in the Cyberduck forums, it's clear that I'm not the only one that thinks this is not a step in the right direction. So I'll bide my time. I'm guessing that there's going to be a compromise here about how the bookmarks are handled. For now, If I get rid of the toolbar, I have only the one I can't get rid of, and that's a reasonable number for me to deal with. Still... too bad.

This morning I read that there was an update to Scribbles - the new drawing app I'm looking at, to v1.2.3. The release notes said something about a bug in one of the brushes, which is nice to know they are paying attention and listening to users. I was able to pick up a nice wireless Mighty Mouse for my MacBook Pro, so I'm hoping to really start to use Scribbles soon, but it's going to depend on the task(s) at had as well. I've got to have something I need to do to really test it out.

I was talking to a few guys around the office today - specifically about how they had not passed a test in their series of certification exams that several more people need to take around this place. I've written about the evils of these vendor exams - written by people that aren't professional educators, graded in the most cursory manner - computerized for the most part, and the resulting problems this makes for people in the workplace that need these certifications for their jobs.

But that's not what I'm writing about now. No, this is something I hadn't realized I'd picked up along the way to where I am today. I'm talking about understanding the difference between "testing" and "fact checking".

What I mean by the first is that anyone, at any time can make a test for someone. We do it all the time as parents ("Did you knock this over?"), we do this as developers ("Is this the problem with the system?") and as companies - in the form of these certifications. They aren't fair, don't really concern themselves with "right" and "wrong" - they are questions, challenges, and tasks that we must overcome to someone else's satisfaction in order to achieve what they have promised we'd receive.

"Fact checking" is all about the "right" and "wrong" - Can a 3kg spherical lead ball dropped from 1m crack a 1cm thick piece of glass? That kind of stuff. It's all something that that is by it's very nature objective. And for the most part, dispassionate.

"Testing" is all about someone's passion - I guarantee it. The person that wrote the material the test is based on... the instructor... the test's author - someone in this chain cares a great deal about this test. And as such, it's not about you, the test taker to tell them what's "right" and "wrong" - they already believe they have a firmer grasp on that than you - which is why you're taking the test and they are administering it.

What surprised me today was that I knew all this, and my co-workers didn't. Then I got to thinking about it and I realized where it all came from - grad school. I have to hand it to grad school, it really taught me this lesson. You sit in a class of less than a dozen grad students, and the professor knows you all by name. He's spending a lot of time trying to make this material understandable to you all, and so there's a lot he's got invested in the exams (tests). You need to make sure that when you take the tests, you give him what he's looking for. Period.

Maybe the university is an easy place to learn this, I don't know. But I do know that I have learned it, and I don't think of myself as cynical on this point. I think of myself as recognizing the concerns, issues, and feelings of the people involved. There's nothing to gain by making the test so simple it's useless as a gauge of the material retained. It's also equally useless to make it so hard that it's nearly impossible to pass - no one will pass it and therefore it'll have no value.

So there's a compromise there - an optimization of sorts. Again, grad school helps me there more than I initially realized. It's taken me 20 years, but I'm really starting to look back fondly to that place. Who knows... maybe we'll take a road trip and see the campus.

Last week I saw an interesting video making the rounds which included a few comments about the legal ownership of data available on the web. For instance, flickr is nice, but in order to put one of my pictures uploaded there on my blog, I have to make a link to the flickr page with that image.

Makes sense from flickr's point of view - it's how they pay for things. But it brings into question - Who's data is is anyway? I mean really... if they can make that rule, and Yahoo's music service can go dark, then what guarantees are there that my images uploaded to flickr aren't going to disappear? Or find their way into someone's photo gallery?

Nothing. Absolutely nothing.

This is one of the reasons I've tended to stay away from these "repository" sites - or hosted source control like SourceForge, hosted Subversion, etc. You never know what your rights are, and what theirs are. Even if you don't plan on having something of value there, are you opening yourself up if you put something up there, later decide to remove it and then try to claim it's really "yours"?

I'm certain there are some cases and some sites where you're out of luck the second the first upload it complete. It's theirs. and nothing you can do is going to change that. You just loose.

With the fact that license/usage agreements can include language that essentially says "subject to change without notice", you can't even be assured that you'll be alerted if the rules of usage change while your data is hosted there. It's almost impossible for the non-lawyer to know what they legally can do and can't do. And saying "Hey, it's a picture of me" doesn't count for squat in a courtroom.

So it's tough for me to see what the best course of action here is. I've decided not to post a lot of things to flickr even though it's a decent service, and could be really useful, but it's that like to the flickr page that causes me to pause.

Skitch is different - they might very well go away, and that would be bad, but I'd be willing to pay got Skitch usage/storage. No question. I might be willing to pay for flickr if they relaxed the rules, but I don't think I can trust that they will.

For the most part, I'm using HostMonster, and that I know I own the data. It costs more, but it's worth it. It's my space and my stuff. I've thought about the dotMac and MobileMe, and if they stabilize it out, I might be willing to go that route, but again, it has to be clear that it's mine for me to really depend on it.

I know someday there's going to be a shake-out and these "free" services are either going to have to charge or be a lot more up-front about the ownership issues. But until that day comes, you have to be personally careful. Better safe than sorry.

I've been in a lot of different business situations in my life - at the bottom, in the middle, at the top, big place, small place, conservative place, educational institution - and still I have to say that I'm amazed when I run into a boss that seems to need to have his people agree with him on a course of action.

Case in point: several years ago I was building an interface to a new ticker source from a group in the larger bank. There was an issue about what data to retain, and I thought that the existing model was fine, but a manager (not mine) in the Shop decided that we needed to retain more - specifically, an additional price.

He explained it and while I understood that I would be forced to do it, I wasn't about to agree that it was a good idea - it wasn't. But that didn't matter. If I had wanted to stay in a position where I was the one making the decisions, I'd have stayed there. I moved for a reason, and this was one of the things I knew I'd have to live with, and I was fine with it. The problem was, he wanted me to think it was a good idea. He just kept at it, and at it to get me to see this as a good idea.

Finally, my manager said "Give it up... Bob's going to do it, but he's not going to agree it's a good idea." He understood. Good.

Today was right along these same lines - even the same manager was involved. They asked me to coordinate all the development of a particular feature. I got all that tested and coordinated the two other guys and they said their stuff was done and tested. So I told one user to kick the tires and see what they thought. Interestingly, there was a difference in the two ways this package was executing the code, so there was a difference to the user. Nothing big, in my mind - we'll get it fixed as soon as the vendor's rep gets back to us about how to make them work the same.

But this manager told everyone we hadn't completed testing on the additions and to hold off testing. Seemed a bit excessive to me, but he's the manager, so I simply wrote back - 'Hey, we tested it, didn't find this, so we'll fix it.' And then the "please agree with me" chain started.

Over the next hour, about half a dozen emails went back and forth about the true state of the testing. I tried to explain that we did test it. But if he wants to be the final tester before anything is shown to the users, then just say so. Yet he continued to want to get me to agree that we hadn't tested it properly.

OK guy, get over it. You want to be liked, then be liked. You want to be the Boss, then be the Boss. But don't expect to be able to be the Boss and have everyone like you. You asked someone (me) to make the call, and then didn't like the call. OK, make the call yourself, but don't expect me to say I made a bad call... I just made one you wouldn't make.

Yikes... just be a bloody manager and say what you want.