What's it all about, Alfie?

This morning, I read this article about a problem in two small towns in Massachusetts when the engineers responsible for a normally reliable, safe, natural gas distribution infrastructure don't really think about what they are doing, and the consequences of their designs. It's shameful.

The NTSB report about what happened was very clear, and very troubling:

The contracted crew was working on a tie-in project of a new plastic distribution main and the abandonment of a cast-iron distribution main. The distribution main that was abandoned still had the regulator sensing lines that were used to detect pressure in the distribution system and provide input to the regulators to control the system pressure. Once the contractor crews disconnected the distribution main that was going to be abandoned, the section containing the sensing lines began losing pressure.

As the pressure in the abandoned distribution main dropped about 0.25 inches of water column (about 0.01 psig), the regulators responded by opening further, increasing pressure in the distribution system. Since the regulators no longer sensed system pressure they fully opened allowing the full flow of high-pressure gas to be released into the distribution system supplying the neighborhood, exceeding the maximum allowable pressure.

Why the control system wasn't smart enough to detect:

• Pressure Reading of Zero - the sensor was on the abandoned line - which had no pressure. Why didn't that fact alone cause an alarm?
• Zero Change on Output based on Input Change - when the first change was done, why wasn't the change seen on the sensor - at least in a percentage measure? Even, if you look at the first-derivative of the sensor reading, a simple thing to do even with analog control systems, and see that no change could be seen should have been an error.
• Upper Limit on Input - when you see that you have increased the input some percentage in the last "...few minutes..." - then sound the alarm. There is no reason to ramp the input like this. It's a leak - or worse, and needs to be corrected.

As an engineer, I'm stunned that the folks that created this control system didn't do these, or something similar, to them to sound alarms as opposed to letting four homes blow up. I mean, it's one thing to have a leak, with a bad scent in the house - but to actually have homes blow up - that's way past bad design.

An old friend and I have talked about this many times over the years - How do you really teach this kind of design? - and it's not easy. It comes from experience, and the problem is, most engineers don't have it - any more than most people are wise. You have to accumulate experience over years - to know what to think about... what to look for... all that needs to be considered, and it takes years to really gain that perspective.

But companies will pass things like this off to the junior engineers - because that's who they believe can do all that's needed. And if you ask the folks doing the work if they are ready to do this kind of work, they are always going to say "Yes!". That's just "confidence".

Until we come to terms with this gap in the education of engineers, we're going to have these problems, and they could be a whole lot worse.

Today I decided to open up some of the fortran code I wrote for my thesis - 30 yrs ago, and see if Sublime Text 3 would handle the syntax highlighting, and therein brought up an interesting point about the problem I had in upgrading from Sublime Text 2 to 3. So it made sense to write it all down just in case it happens on another upgrade in the future.

I found a nice Fortran package Sublime Text 3 - it did all the normal syntax highlighting stuff, and lots more. Very slick that it picks out the LaTeX formatting in the comments - Who thinks of this? Wild! So it seems like a great little find. Sweet.

I go to install it, using Package Control, and I noticed that Package Control wasn't installed. That's odd... I know I had it installed - maybe it didn't come over in the upgrade? So, let's install it again... No biggie.

Well... kinda... When I following the installation instructions, I didn't see it show up in the pop-up. Very odd. I checked the directory structure - it was there... but it wasn't showing up. Very odd...

So I Googled the issue, and it seems that it's something that happened enough to warrant a tip - it might be disabled. Go to the general settings, a JSON file, and then see if it's in the list of disabled packages. Sure enough - there it was! I simply deleted it from the config, and restarted Sublime, and Bingo!

After that, everything was working just fine:

I know we all just want to have some sense of satisfaction in our lives. To feel that we matter - somewhere. To feel that we make a difference. It's universal... but so many don't have that because of circumstance or society. But that's another issue. What we, in the US Tech Industry have created is a class of people that feel entitled to things that they have no entitlement to. My sister calls these The Soccer Trophy generation.

If someone is told - at every possible turn - that they are amazing, valued, better than the rest, then their sense of reality is being formed in those moments. If all you allow a person to see is a mirror, reflecting back to them that they are the center of the world, then they have no opportunity to see what reality is like outside those mirrors. It's not surprising then, when these people get angry when the context is changed, and it's time to work together with others of differing opinions, and then all of a sudden, things go very badly.

Over the last five years, I have been working with folks that fit this profile perfectly. They are fine people - at least in comparison to most federal inmates. But they are employees that feel the company owes them a sense of purpose, a job that fulfills them... and they have every intention of getting that. But that's not how the world really works. Sometimes it's just a lot of hard work for very little short-term payoff. Life isn't fair. But telling this to our little monsters usually creates such a stink that it's avoided at all costs.

I'm all for being nice. I'm all for being generous. I think these are fantastic qualities, and I try to be as graceful as I can be. All the time. But some days I just want to say "Someone did you a disservice by making you think the world revolves around you. It doesn't." Now let's have a little moment to deal with that, and let's get back to work!

But that's not done... Go figure.

I was updating my macOS 10.14 Mojave install to get Apache + PHP + Postgres working, and I realized that I probably never wrote up this great Postgres client for macOS - Postico. When I saw it, and how complete and efficient the UI was, I knew I had to buy it - just to support more development like this.

Normally, I'm a psql guy because I'm far faster on the command line than I am in a GUI - it's just the number of hours and the fingers are always on the keyboard. Plus, let's face it - the feature set of psql is very impressive - I've got it in scripts, and you can pass in variables for insertion into your scripts... it's just pretty amazing. But still... great design needs to be supported, and this is one that really needs the support.

There are so many things I like about this UI. First, it's efficient - you have a query window, and then a table of results. I wrote something like this many years ago, and I had the same UI - but I stopped there. They didn't, and I'm glad they didn't. They then filled in the window title bar with the connection information and the status of the connection. They even let me know if it's local or remote. Lots of good stuff there at a quick glance.

Then we can look at the footer of the window - we can export this data in a variety of forms, and it lets us know the row count, if the query was successful, and the time required. This is all great stuff, and it doesn't chew up a ton of screen real estate, but it's all there - efficient

There is a lot more to this guy - as a Mac app, you'd expect multiple connections to different databases - and you get that as well. I've never really needed it - but again, I'd just open up several terminal windows and then run psql in each. But it's great to know that they spent the time to get all the details right.

Anyway... maybe I'll use it a lot - someday... but I love supporting great designs, and this is a really good one.

This morning I thought I'd perform the ritual of getting the old web development tools that I've used in the past going again - this time on macOS 10.14 Mojave. Now I haven't used PHP in ages, but I've still got code and databases for Postgres to use that - so it makes sense to get this all working again, and it's always fun to see how things work out.

Getting PostgreSQL 10.3

Loads of coverage here about Postgres, and it's just so simple to get the latest version from Homebrew:

  $ brew install postgresql

I've even posted how to upgrade from major version differences, so it's easy to get the latest Postgres running on your box, and the tools are just superb.

Activating UserDir in Apache 2.4.34

As in the previous updates, the UserDir extension is not enabled by default, so we need to get that going right away. This enables the code to be run from the development directories, and that's a big time-saver. First, we need to enable the UserDir module in Apache, and then make a specific config file for the user in question. Start by editing /etc/apache2/httpd.conf and line 174 needs to be uncommented to read:

  LoadModule userdir_module libexec/apache2/mod_userdir.so

and then similarly on line 511 uncomment the line to read:

  Include /private/etc/apache2/extra/httpd-userdir.conf

Next, make sure that the file we just included is set up right for including the user
directories. Edit /etc/apache2/extra/httpd-userdir.conf and line 16 needs to be uncommented to read:

  Include /private/etc/apache2/users/*.conf

At this point, you need to make sure you have at least one file in the /etc/apache2/users/ directory for each user, like: drbob.conf:

  <Directory "/Users/drbob/Sites/">
      Options FollowSymLinks Indexes MultiViews ExecCGI
      Require all granted
  </Directory>

where the last line - Require all granted is new as of Apache 2.4, and without it you will get errors like:

  [Thu Dec 18 10:41:32.385093 2014] [authz_core:error] [pid 55994]
    [client fe80::7a31:c1ff:fed2:ca2c:58108] AH01630: client denied by server
    configuration: /Users/drbob/Sites/info.php

Activating PHP in Apache

The mext thing to do is to activate PHP in the supplied Apache 2 with macOS 10.14. This is line 177 in the file - /etc/apache2/httpd.conf and you need to uncomment it to read:

  LoadModule php7_module libexec/apache2/libphp7.so

and then verify a file called /etc/apache2/other/php7.conf exists and contains:

  <IfModule php7_module>
    AddType application/x-httpd-php .php
    AddType application/x-httpd-php-source .phps
 
    <IfModule dir_module>
        DirectoryIndex index.html index.php
    </IfModule>
  </IfModule>

which does all the other PHP configuration in a separate file to make upgrades easy.

Finishing Up

At this point, a simple restart of apache:

  $ sudo apachectl restart

and everything should be in order. Hit a URL that's a static file with the contents:

  <?php
    phpinfo();
  ?>

and you should see all the details about the PHP install - including the PostgreSQL section with the version of Postgres indicated:

What's really great is that Apple has included lots of support in the default PHP install:

• PHP 7.1.19
• Postgres 9.3.7
• MySQL 5.0.12
• SQLite3 7.1.19

so there's no reason to do anything more to get the kind of support that I used to get. And I get the other databases for free. This is great news! I then run my little test page to make sure the database ocnnection is working:

and everything is working exactly as expected!

This morning I wanted to make sure that Homebrew was up-to-date on my laptop - now that I'd upgraded it to macOS 10.14 Mojave. In the past, changes to the OS and Xcode almost always meant that I had to update with a simple:

  $ brew update

and off it would go - updating the code for brew as well as the list of new, updated, and deprecated packages. This time, it also pointed out that in the upgrade to Mojave, I think Apple cleared out some empty directories. The update instructed me to:

  $ sudo mkdir -p /usr/local/sbin
  $ sudo chown -R $(whoami) /usr/local/sbin

so that the /usr/local/sbin directory was created, and owned by me. Sounds fair - I don't think I had anything there - and we're good to go, but it's really impressive how they have handled all the package management and versioning. I really like the tools these guys make.

I read a tweet this morning about Apache Pulsar achieving top-level project status in the Apache organization. I hadn't heard of it, so I spent a little time reading about it and getting familiar with the differences between it and Storm, Flink, etc. What I found was something that I might actually use in the future.

The important parts are that Pulsar is a message broker with reliable messaging, with storage and fail-over. This makes it way better than many other systems, but it goes a lot further than that. Because it's Apache, it's using ZooKeeper, but it's also using BookKeeper for the message storage - which is an interesting twist.

It's scalable up to a million topics... is meant to sync servers by geo-location, and all the goodies you'd expect from something like Cassandra or the like. Lost of resilience. Turns out, it was the message broker for Yahoo, and has been open sourced. It's low-latency, and meant to be server-to-server in the datacenter, even if they are separated by continents. So it's battle-tested. And all this makes it a useful project because there are lots of times I can imagine needing a better broker than SQS.

But what makes it even nicer is that it has all the Lambda capabilities that AWS has as well. So now you can add in the streaming capabilities of Storm in the messaging system - as opposed to having a streaming system talk to Kafka as a source and sink. This is a different take on the problem, but one of the problems I had with Storm was the scaling of the nodes - it was hard to get them to not back-up. If this is a message broker, then we have a lot more flexibility on the processing time, and every hand-off is to a queue/topic. That's good news.

I'm not sure if/when I'll need to have something like this, but I'm very happy to see that they have it, and it's gotten to a point that it looks pretty stable. Good news!

Withe the recent improvements in performance from Xcode 10 for my Swift code, I thought I wanted to dust off DKit and make sure it still works on macOS 10.13.6 with Xcode 10 - it's been quite a few years since I've made any changes to it, and I just wanted to make sure it all builds and runs.

One of the things I remembered from a few months ago, when I tried to dust it off for a homework project for a job interview and I had some problems with the fact that the shared library was being built as a dylib and that just makes it a lot harder. At the time I originally wrote the library - some 15 yrs ago - there wasn't the support in Mac OS X for the standard unix SO shared library - so I had to use the dylib and figure out how to link it in properly.

Today, the story is entirely different - and honestly, for several years, I've known that macOS could generate the unix SO shared library. So I just switched over to that this morning, and gave it a spin. Turns out it worked perfectly.

I was able to compile all the components of the library, and then the test apps, where the templates are really being used, and all the tests ran. I checked in the simple changes to the Makefiles and everything is solid.

I'm sure there are a lot of things that this could add - I could make a test app that read a file and sent it out on a TCP or UDP transmitter - for the testing of the TCP and UDP receiver, but I'm not worried about that right now. It's enough that it's still working just fine, and builds on the latest macOS. Can't beat that!

Linus Torvalds is in the news again this morning, and I'm glad to see he's now aware of the effect he had on people, it got me thinking about git and I thought I remembered that he wrote the first version of git after something happened with the license on the SCM tool he was using on the Linux Kernel. So I hit Wikipedia, and sure enough, there was a license issue with BitKeeper, and so Linus wrote the first version of git.

What I hadn't remembered was the timeline - and that's what amazed me this morning:

• 3 April 2005 - Development Starts on git
• 6 April 2005 - the git project is announced
• 7 April 2005 - git is self-hosting
• 18 April 2005 - first merge of multiple branches
• 29 April 2005 - performance target is achieved: 6.7 patches per second
• 16 June 2005 - linux kernel 2.6.12 release is managed by git

So a couple of months to get a SCM that hosts one of the most influential codebases that currently exists... that's more than impressive... that's down right amazing.

Think about all that git has effected... GitHub, BitBucket, GitLabs - there's an industry built up around it - that's wild to think about. Sure, the guy may not be someone you want to have over for dinner, but what he accomplished in a few months was nothing short of amazing. And all of us in the tech community have benefited from it.

There is something to be said for talent... it's not necessarily sitting in the prettiest of packages, but we would all do well to look past the packaging, and try to see the real value under the covers. It can be revolutionary.

I've just finished a book that I really enjoyed, but I'm glad I read it last of the three - in fact, I'm glad I read each of them in the order in which I did. It made for a much smoother reading experience - because I started with Pragmatic Scala and this was really based on the fact that we are using Scala in The Shop, and as such, I felt it was important to get up to speed on the language, and the Pragmatic Books have always been pretty good, in my experience.

Well, this was a fine book on Scala, but I have to say, I'm stunned that Scala is as popular as it is. I understand that one of the key design goals of the language was to be a pure Object Oriented language - and as such things like static methods and variables are not allowed in the instance variables, but they solve that with partner classes, and that's OK... but it is confusing for many folks - simply because most coders are not going to have a Theory of Languages in their past.

Still... Scala is a language we use at The Shop, and that means for better or worse, this is something that I needed to know. The next book was a little more fun because it was all about Xcode. In Xcode Treasures, the author walks through the tool not the SDK. This is nice, because in my recent work, I've re-written my crypto quip solver to Swift 4.2, in Xcode 9, and the times are really not horrible - compared to ObjC and Ruby, but Clojure still wins the speed race, but that's to be expected.

And while I spent plenty of time in Xcode quite a while ago, it's nice to see how much progress has been made in Xcode recently. The handling of assets like icons and images, and being able to load them without worrying about the resolution is really nice. Also, the entire Gatekeeper and Code Signing is now simple, and it used to be a pain. That's a great relief.

So this was a really good chance to catch up on a lot of the capabilities with Storyboards in Xcode and the scripting and such... very nice book. The final book was Mastering Clojure Macros and this is one I've been trying to get through for quite a while - many months, in fact. It's not a long book, but it's dense, and it takes time to make sure you understand the concepts because macros in Clojure are hard enough, and some of the examples are compact - as Clojure is want to do, and that just makes it all that much harder.

But the book was just amazing! What a great study of the subject. This was one of the few topics I really wanted to get better at with Clojure. Yes, I'd like to get a little more into spec, and core.async could be very useful if you're making small, independent apps, as opposed to big, multi-host apps where you typically share messaging queues, etc., but still... macros are in everything when you really get into things and I have been able to do quite a bit with them to date, but I wanted to know more.

Of the three - they are all worth reading - if you want to learn the material, but I really enjoyed the last two far more than the Scala book, but that's because of the subject - not the book.