What's it all about, Alfie?

I know it's ranting... I know it's tilting at windmills, but darn it, these large ISPs that are not really doing the right thing for their customers are really yacking me off! The other day it was Comcast and their blocking ports and messing with responses to requests. It's frustrating that it seems the companies with the capital to provide large bandwidth are the same ones that are the most intent on keeping it to web and mail traffic. It's incredible.

Then today I'm trying to upload a few files from work via an AT&T DSL line to my HostMonster account and it starts to send but fails in the response. I know about this as it's happened before, and I can solve this problem by doing it while I'm at home - from Comcast. Now I don't know why AT&T's DSL is messing things up - and it honestly might not be AT&T - but it's upstream where I can't see the HostMonster boxes and they can't see me.

It's really amazing. In the old days, I'd have had a modem in my laptop and I'd dial into the ISP I needed to get to and things would just work. Today the speeds are faster - which is nicer, but the connectivity is probably a lot less. Trying to stretch their infrastructure to the limit to get the most customers on it before having to upgrade it is just the typical corporate way... and it's leading to regulation. They are going to find themselves looking down the business end of massive regulation if they don't clean up their act.

One of the things that I keep coming back to every so often is the simulation work I did during my Ph.D. Now that I had upgraded to 10.5.1 on my laptop, I wanted to see about getting the code going in x86_64 mode. I have had it running in 64-bit on PPC, but now that Xcode supports it, and the gfortran I use does as well, I was interested in seeing if it was going to run faster - or slower, in x86_64 mode.

The first thing I noticed was that the LINPACK routines that I had taken and hand-tuned to the problem were not working out well with the 64-bit compiler. I was getting SegFaults, and rather than mess with trying to fix those versions, I thought I'd use the BLAS and LAPACK that are bundled with Mac OS X in the Accelerate Framework. These are supposed to be optimized for the AltiVec (PPC) and SSE3 (Intel) so I was thinking that moving this way was a nice upgrade.

The code changes weren't major - primarily in the data storage going into the functions, so it only took me a few hours to fix all that up and clean up the code with a few #ifdefs to make it compile either with the LINPACK routines I built or with the LAPACK functions that came with the OS. What was major, were the results.

As I had hoped, the 64-bit version of LAPACK was faster than the 32-bit version. However, the surprise was the fact that my 32-bit hand-optimized routines were faster still. If I wanted, I'm guessing that I could update these guys to 64-bit by looking at the use of the data element sizes - that's got to be the cause of this as the logic is fine, and then I might very well have something that's faster still. What a shot in the arm! I had no idea that the modifications I had made were going to be that fast. Good for me.

UPDATE: I went into the code and found that it was a simple matter of how the integers were being passed from FORTRAN to C. By putting a simple typedef in the code:

    /*
     * Because we need to be able to build this for 32-bit and 64-bit
     * versions, I want to be able to typedef the integer here so that
     * the value coming in from the FORTRAN code matches what we will
     * use here. Without this, we'd have a mess on the conversions.
     */
    #if defined(__x86_64__) || defined(__ppc64__)
    typedef int f_int;
    #else
    typedef long int f_int;
    #endif

and replacing the long int with f_int (FORTRAN int), I was able to use the same code for both builds and the errors went away. Nice.

Since it is so new, I didn't want to turn on TimeMachine too early in my using Mac OS X 10.5 (Leopard) because I didn't know if I was going to have issues with 10.5 in the first place, or exactly how they built TimeMachine and how it would function. My initial questions were pretty basic: What would happen if I disconnected the drive between backups? Would it error on me when it tried to run? How fast was it going to be? Basic things that you want to know before you invest a lot of time into using a tool like this.

I was initially pretty sad to see that the initial backup of my MacBook Pro was 64.64GB, and took about 3.5 hours on a FireWire 400 external drive. Yeah, if I had a FireWire 800 drive it would have been faster, but this is the guy I got a while back to do my own backups on. So it's not fast for the first (initial) backup. But it seems to be complete. The real question to me insofar as 'completeness' is: Does it backup /usr/local/ as well? I'm going to have to dig into it and see what it's got. Maybe it does, and maybe not. Certainly, it's not a crime if they don't, but then I'll have to maintain my own backups of /usr/local/ when I install new things, etc.

When it was done with the initial backup, I was pleased - nay, very pleased, to see that if you unmount (disconnect) the external drive, then TimeMachine sees this and says that the next scheduled backup is when the drive is connected. Very nice, guys. Very nice indeed. This means that I can use it as I want - keeping it connected when I'm at work - most of the time, and then when I roam, I'll be without backup or restore. Not a bad trade-off, really.

I'll have to see how it runs for the rest of the day taking backups every hour. I'm hoping that I don't even notice them being done. That would be ideal. On my G5 iMac at home where I've had TimeMachine running since I upgraded it, there's a noticeable activity on my LaCie 500GB drive, but that's the drive and not the machine or TimeMachine. I can live with that.

So far, very nice. We'll see how it runs, now.

UPDATE: it's great to know that the backup includes /usr/local/ - that's a big load off my mind. Super.

When I put code segments into a post, I've used BBEdit's Copy as XHTML function to get it out of BBEdit and into the post. It makes something that's impossible to edit in the post, but it looks nice, and it previews nice in MarsEdit. So, all-in-all, not a bad solution.

But it's not perfect.

So I decided to look for a WordPress plugin that would do a similar job, but simply, and without a ton of things I had to manage/deal with. What I found seems to be a nice little compromise between what I had and what should work really well for me. It's called WordPress:CodeHighlighterPlugin (obvious, yes) and it appears to do all the languages I can imaging and it's all just PHP code as a plugin to WordPress. Additionally, to use this you simply use the <pre> tag with an additional lang argument to indicate which language to use. You can also optionally specify line numbers to be generated and where these line numbers should start.

While this plugin doesn't allow me to preview the code in the form it will appear, it does allow me to easily edit it. So it's a trade-off. I'm not sure how much I'll use this guy, but it's loaded and it's there, and should I need it, it's nice to know that it's there. I guess I'll try it out the next time I need to put code into a post.

There's a reason I like working where I do - it's not the hours, that's for sure. It's that for the most part decisions are made by the appropriate people. That means that if there's a need for the entire organization to be involved, then the decision is handled at a very high level. However, if it's the decision to update from an unsupported product to a supported version of the same thing, then, again, for the most part the decision is handled at a low level as there's not really a lot that we're going to do about it. We're going to have to update, and what we update to is really the only question.

Today I've had a couple of things I'm working on go from being properly handled to being very improperly handled. A while back it was the ordering of hardware and the request to get the vendor's technical sales reps involved. True to my prediction, they knew nothing. They took more than two weeks to come back and tell us that they knew nothing. Total waste of time.

A few weeks ago, we wanted to get a nice messaging bus in - Tibco EMS. We knew it was going to be a ton of money, but there were a ton of reasons to spend it now and convert the Shop to Tibco. But since it wasn't going to solve a pressing problem, it was into the "project" category... complete with timelines, Visio diagrams, step-by-step proposals. The Works!

I read a wonderful quote the other day attributed to Mark Twain: The best time to fix a hole in the roof is when the sun is shining. The reality here is much more like: They only time to fix the hole in the roof is during a monsoon. I appreciate that this is not pocket change that we're talking about, but the problems facing us aren't likely to be solved in an afternoon, either. Serious problems demand serious people and serious tools. This is not something to slap together with perl and Visual Basic.

But that battle is lost. Today I was trying to get a new version of the modeling libraries into production. I try to keep a low profile - there's no real option. We have to do it, so the fewer people that know about this the better. The fewer people involved the better. More visibility almost always translates to slower decisions, more arguments, more grief, and in the end, since there's really no other option, simply a slower roll-out of the needed functionality.

I tried not to get flustered with this as it was a friend that made the critical mistake of involving too many people early on. His opinion is (was) that this would be 'no big deal', and could not have been more wrong. I appreciate what he was trying to do. He just wants to keep people up to date and he thinks it'll be a slam dunk. But when someone usually thinks that I have the nervous feeling that if they are wrong, they'll be wrong in a very big way.

So it got involved with four more people that have very little time to look at things and have very little idea of the real alternatives open to us. This means that they'll look at a difference of 0.0005 and say "That's too much - match it exactly!". But Dude! That's the old numbers... we match the new ones exactly. Let's not get bogged down about the difference on what was - let's look at what is.

But I know that's not how it's going to go. It's going to be a few days of people discussing it and trying this permutation on the data, and that input set, and this and that until they get frustrated and finally see that it's the old values that are off and give up. In the end, it'll be just as if they never got involved. They'll add nothing, because there's nothing to add. But it's something to argue over.

This the is most Dilbert-like part of my job. I really don't like it.

There are days that I don't mind that Comcast is a big organization that really is horrible at customer service and responsiveness to their customer base. Then there's the P2P spoofing that's annoying, but for the most part, even this is glossed over in my day-to-day interactions with them. After all, there aren't a lot of choices in the broadband market in the Chicago area. But then today happens and it really starts to make me angry.

I was trying to simply upload a few web pages to my home servers - nothing special, but I wanted to upload them and then look at them. But Comcast's monitoring of the 'standard' ports and hindering them got in the way. See, because I have multiple machines with the same servers (web, ssh) running, I needed to do port forwarding on the firewall, so only one machine is on the 'standard' ports, and the others are on clearly non-standard ports, but since I know what they are, I can deal with it.

So I try to SSH to a non-standard machine and I get in fine. I try the same thing to the standard port machine and Comcast times out on me. I go to the non-standard machine, get in fine, and from there, SSH to the standard machine just fine. So the machine is not the problem - it's bloody Comcast and their crappy port hindering/filtering/dropping rules. Why? What does it matter if I use the capped bandwidth for web serving or downloading stuff from their web site? Who cares? Are they trying to legislate/regulate morality? Computer usage? What gives?

I try the same thing with the web servers. The standard one fails and the non-standard one gets in just fine. So I'm sure that it's Comcast, and they have a record of the P2P problems and now that's a class-action lawsuit. When will they learn? Be the "Good Guys", and just let people use what they have paid for. Don't waste your time on the filters, spoofers, lawyers, etc. Just sell us the service and let us decide how to use it.

Do the phone companies regulate how you use your phone? Nope. How about the water company? Nope. Why should we allow these crappy ISP companies to do it? Because most people don't know or care.

Grrrrr... makes me mad.

Today I've started really working with Leopard (10.5.1) and there are a few things that aren't really exactly how I'd like them to be. For the most part, I'm willing to let these things slide, but there are a few that are somewhat annoying and others are nice surprises.

Don't get me wrong... I know 10.5 is what I need to go to, and it's clear that there has been a ton of thought put into it, but there are just a few rough edges that I'm sure will get worked out in the coming months. I guess I'm just realizing that the critics of Leopard are, in part, right. It does seem a tad rushed. But given all that's happening, it's clear that this was easier to rush than the iPhone, so they put their limited resources where they needed them. It'll just take a while to get everything back to working really well.

SSH

One of the nicest things about Leopard is the integration of SSH. Specifically, the addition of keys and ssh-agent to the login process. In the previous versions of Mac OS X, I had to create a script for my .login and .tcshrc that starts ssh-agent if it's not already started, and if it has been started, reads the environment variables from the output of that command - stored in a file in /tmp/. This lead to problems if someone ran a new shell and the .login script didn't get the environment variables right, so it'd start a new ssh-agent and blow away all my old env vars. A pain.

Now, when you first ssh to a host a dialog box pops up asking you for your SSH pass-phrase, and if you want to store it in the KeyChain. I put it in, said "Yes", and now all my ssh sessions are authenticated properly without me having to worry about a thing. This is exceptionally nice. Saves a lot of hassles for me as I use ssh all day long.

Spaces

Prior to 10.5, I was using VirtueDesktops - an open source virtual workspace app for 10.4. It was not perfect, but it worked, and as long as it was working for me, I was happy. I knew with Spaces coming, this app was at an end, and that's the way of the world. But there are many things in VirtueDesktops that Spaces is missing. Some minor - but some major.

First off, Spaces only allows three ways to switch spaces. It's really just allowing you to choose one of three modifiers and a numeric key for going directly to a space. There are also similar modifiers to the arrow keys to move from space to space, but since I use BBEdit, the keys I would have used are already used in BBEdit, and so I had to turn that off in Spaces. Not horrible, but annoying that I can't use Fn keys as opposed to numbers.

Secondly, the automatic switching to the app upon selection. That's a nice feature - if you want it, but I don't, and the problem is that when I want another Safari window, I get thrown back to the space with Safari open, and that's very annoying. I have to make a new window there, then F8 to see all the spaces, and then drag the new window to the space I want it, and then switch back to that space. Non-ideal.

Thirdly, the animations for switching from space to space - I'd like to turn it off. You can turn off the Dock's magnification... and the window closing effect... so why not this? No reason, but you can't. This is only annoying, but when I'm moving from one space to another the last thing I need is eye-candy. Make it optional, guys.

X11

There have been a lot of things written about this already. It's not really all there, and it's primarily due to the fact of moving off the XFree86 codebase to the xorg codebase and the differences therein. It's nice that they made it start with launchd, but it needs to read the .Xdefaults properly, and there are still a lot of things that seem a little odd about it.

For instance, I know the app in the Dock is really /usr/X11/X11.app and to change the app icon, you need to change the permissions on the app bundle and then change the app's icon. I did the following:

  cd /usr/X11
  sudo chgrp -R admin X11.app
  sudo chmod 775 X11.app

I then went into the Finder and changed the icon like you would for any application, and then locked down X11 again with the following:

  sudo chmod 755 X11.app
  sudo chgrp -R wheel X11.app

A few times - right after the update, I had a few problems getting X apps to display on my MacBook Pro from my Sparc 20. The problems went away and it could have been the 10.5.1 update, but it's still a little troublesome.

Overall, however, X11 is far better in Leopard than Tiger. The launchd aspect was really a beautiful solution.

Time Machine

I'm so unsure of the effects of this, I've only got it running on one machine, and it's not a critical one. When I've seen it run for a while I'll look to turning it on for the other machines I have, but for now, this has too many unknowns for me to feel comfortable trusting right now.

Xcode

One of the really nice things that's in this release is Xcode, and GCC, in general. There are things that have changed, and while I know to expect them, it's really surprising how easily I found them.

In CKit, I use a #ifdef for defining MAXHOSTNAMELEN as it was defined on Solaris and Mac OS X easily enough, but Linux seemed to be hiding it somewhere. I put it in and ran with it. No big deal. But with 10.5.1 and Xcode 3.0, it was missing from the #includes I was using. So I used this occasion to dig a little further. It turns out that it's located in sys/param.h for all three OSs, and I was just missing it in the #include-chain on linux. No big deal. I take that out, put in the include for that file and we're good to go.

Since I do a lot of coding to x86_64 on opterons, I was very happy to see that architecture supported on the latest Xcode. While much of the code I write is specific to linux, I make significant efforts to make my CKit library as cross-platform as possible, and this certainly means 32-bit and 64-bit on Mac OS X as well as Linux and Solaris.

This weekend I spent the time to update my machines to Mac OS X 10.5.1 - Leopard. The updates went well - both PPC and Intel, and while there were a few issues, there wasn't anything that seemed to be systemic about the problems. After the install, I immediately updated to 10.5.1 and that went well. I had to update a few applications after the long install, but those went smoothly as well.

Spaces is nice, it's not the hot-key combination that I'd like, but I can live with it. It's only going to take a little time to get used to the different key combinations. The UI is nice. It's different from Tiger, but there's nothing really horribly wrong with it, either.

There is going to be a time when some applications are not performing 100%, but that happened with Tiger (10.4) and it'll only take a little while to get them sorted out. All-in-all, it was a lot of work for five machines, but it was worth it.

UPDATE: I needed to reconfigure ssh in launchd on sherman (my G5 iMac) to make sure that it answers to port 2224 as well as 22. This is because I'm using that port to redirect it through my router.

Today was a little mixed bag of coding issues. I had some old code to clean out in the server for option strike formatting that was causing me grief, and then after yesterday's problems with my Sun development box for the MareketData server, I decided to make sure that I could build it on another box - the other development Sun box that my co-worker has.

For the most part, it was building pretty nicely. I had to fix up a few things and move a few directories to a new partition to make space for them, but other than that it was humming along pretty nicely. Then I started noticing some stalls in the network traffic and did a little netstat -i to see that there were some in-bound errors. The network crew replaced the cables from the box to the switch and I'm hoping that will help, but I really think it might be a duplex issue on these auto-negotiation NICs and switches. We'll see.

There was also a mount point issue, and after I pointed out the problem they updated the YP maps so that after the nightly push, I'll be able to reboot the box and then get the right maps for the box. With this, I should be able to build the code on the other box. That will certainly give me a nice buffer... some breathing room, so that if my development box goes on the fritz again, we have some place to build the code for a production problem like we had yesterday.

I also added in some interesting little code to the MarketData server to allow all the symbol mappers for each of the data providers (each provider has it's own symbology, and these components map from our symbology to the native symbology of each provider's API) to change their mapping parameters. For the most part, these are database connections where I run stored procedures or SQL queries to get the native symbology components from our symbology components.

What this is going to allow me to do is to change the database connections on the fly through the chat interface of the server. I can also query the server and see what parameters are being using by each provider. This would have made the need for a rebuild unnecessary yesterday, and it makes switching as easy as typing in a few lines to the chat bot for the app. Not bad.

We also gave a presentation about the existing infrastructure and why we want to use Tibco EMS as opposed to the home-grown message brokers that we have. It was basically showing the other folks - including management, how much we depend on custom communication protocols and why it'd be a good idea to standardize on something like Tibco and get the non-standard stuff out of the shop. It's going to be a hard-sell, unfortunately, as it still represents a ton of money to do the switch. When it's important to them, they'll find the money. Until then, I can make what we have work.

Yesterday morning I was on the way to the train station and my car's CHECK ENGINE light came on. Right in the middle of a turn. So since I don't believe in ignoring problems, I took it to the local shop that had worked on it less than 10 days ago to fix the same CHECK ENGINE light issue. At that time, there were hard errors from the computer saying misfiring of #6 cylinder, so they did a complete tune-up and everything went away. At the time, I was happy to get it cleared up.

Then yesterday morning happened. I was not happy.

Thankfully, they checked it out quickly and today they said they could not find anything from the computer and the CHECK ENGINE light had gone out. So it's crying wolf, and messing with my head.

I have no desire to drive my car if there's something really wrong with the engine as I depend on it far too much to have it say "Hey, buddy! I've got a problem here!" and ignore it. But at the same time, it's a major hassle to do without my car for a day each time it decides to throw up a CHECK ENGINE light with no real underlying cause.

Hopefully, they'll be able to track it down. Otherwise, it may be a continuing problem for me. I'd hate to have to go back to the dealership, but if necessary, that's what I'll do.

UPDATE: they looked and test drove it and nothing ever came up. This isn't good, but it's better than it blowing up on me on the way home. Alive is better than dead.