What's it all about, Alfie?

The use of async/await in Javascript is a nice way to make traditional Promise-based code more linear, and yet for the top-level code in a Node script, await can't easily be used, because it's not within an async function. Looking at the traditional top-level script for a Node/Express project, you would look at bin/www and see:

  #!/usr/bin/env node
 
  // dotenv is only installed in local dev; in prod environment variables will be
  // injected through Google Secrets Manager
  try {
    const dotenv = require('dotenv')
    dotenv.config()
  } catch {
    // Swallow expected error in prod.
  }
 
  // load up all the dependencies we need
  const app = require('../app')
  const debug = require('debug')('api:server')
  const http = require('http')

which starts off by loading the dotenv function to read the environment variables into the Node process, and then start loading up the application. But you can't just toss in an await if you need to make some network calls... or a database call.

Sure, you can use a .then() and .catch(), and put the rest of the startup script into the body of the .then()... but that's a little harder to reason through, and if you need another Promise call, it only nests, or another .then().

Possible, but not clean.

If we wrap the entire script in an async function, like:

  #!/usr/bin/env node
  (async () => {
    // normal startup code
  })();

then the bulk of the bin/www script is now within an async function, and so we can use await without any problems:

  #!/usr/bin/env node
 
  (async () => {
 
    // dotenv is only installed in local dev; in prod environment variables will be
    // injected through Google Secrets Manager
    try {
      const dotenv = require('dotenv')
      dotenv.config()
    } catch {
      // Swallow expected error in prod.
    }
 
    // augment the environment from the Cloud Secrets
    try {
      const { addSecretsToEnv } = require('../secrets')
      await addSecretsToEnv()
    } catch (err) {
      console.error(err)
    }
 
    // load up all the dependencies we need
    const app = require('../app')
    const debug = require('debug')('api:server')
    const http = require('http')

While this indents the bulk of the bin/www script, which stylistically, isn't as clean as no-indent, it allows the remainder of the script to use await without any problem.

Not a bad solution to the problem.

Like many folks that are developers, I'm very interested in what the new 16" MacBook Pros with Apple Silicon will be like. Right now, I'm really focusing on a few key features I've read, and seen, on the 13" MacBook Pros with M1 chips: the Display handling, and the Operating temperatures.

Right now, I've got a couple of LG 5K monitors, and my 16" MacBook Pro can drive them, but the temperature of the laptop starts to rise, and then the kernel_task rises, and the box essentially becomes unusable. I understand all the reasons for the kernel_task, and how it keeps the machine from overheating. And I've blown out my recent-vintage 16" MacBook Pro, so it's not an obvious issue, but I get it... I'm driving big 5K monitors, and that heats up the laptop.

From what I've read, the new Apple Silicon MacBook Pro can drive one 6K monitor, or two 4K monitors - just like the new Mac mini. Which is nice, and I'm just betting that the new 16" MacBook Pros will be able to drive two LG 5Ks - like the current lot can. Which will be nice. And to have no fan noise - that will be the real treat.

Which comes to the point of the thermal environment for the new laptop. I get that they need to be able to cool the components, but I'm to the point that I don't Zoom or run Google Meet on the laptop as it just makes it too hot. I can run Zoom and Meet on my iPad Pro, and it's faster, better, quieter, and I don't have to worry about a long meeting contributing to my machine slowing down.

I know it'll be the end of this year to get the new machines, but at least the new iPad Pros will be out sooner than that - and that too, will be a very nice upgrade. 🙂

Today I've been working on some code for The Shop, and one of the things I've come to learn is that for about every feature, or service, of AWS, Google Cloud has a mirror image. It's not a perfect mirror, but it's pretty complete. Cloud Storage vs S3... Tasks vs. SQS... it's all there, and in fact, today, I really saw the beauty of Google Cloud Tasks over AWS SNS/SQS in getting asynchronous processing going smoothly on this project.

The problem is simple - a service like Stripe has webhooks, or callbacks, and we need to accept them, and return as quickly as possible, but we have significant processing we'd like to do on that event. There's just no time or Stripe will think we're down, and that's no good. So we need to make a note of the event, and start a series of other events that will to the more costly work.

This is now a simple design problem. How to partition the follow-on tasks to amke use of an efficient loadbalancer, and at the same time, make sure that everything is done in as atomic way as possible. For this project, it wasn't too hard, and it turned out to actually be quite fun.

The idea with Cloud Tasks is that you essientially give it a payload and a URL, and it will call that URL with that payload, until it gets a successful response (status of 200). It will back-off a bit each time, so if there is a contention issue, it'll automatically handle that, and it won't flood your service, so it's really doing all the hard work... the user just needs to implement the endpoints that are called.

What turned out to be interesting was that the docs for Cloud Tasks didn't say how to set the content-type of the POST. It assumes that the content-type is applciation/octet-stream, which is a fine default, but given the Node library, it's certainly possible to imagine that they could see that the body being passed in was an Object, and then make the content-type applciation/json. But they don't.

Instead, they leave an undocumented feature on the creation of the task:

  // build up the argument for Cloud Task creation
  const task = {
    httpRequest: {
      httpMethod: method || 'POST',
      url,
    },
  }
  // ...add in the body if we have been given it - based on the type
  if (body) {
    if (Buffer.isBuffer(body)) {
      task.httpRequest.body = body.toString('base64')
    } else if (typeof body === 'string') {
      task.httpRequest.body = Buffer.from(body).toString('base64')
    } else if (typeof body === 'object') {
      task.httpRequest.body = Buffer.from(JSON.stringify(body)).toString('base64')
      task.httpRequest.headers = { 'content-type': 'application/json' }
    } else {
      // we don't know how to handle whatever it is they passed us
      log.error(errorMessages.badTaskBodyType)
      return { success: false, error: errorMessages.badTaskBodyType, body }
    }
  }
  // ...add in the delay, in sec, if we have been given it
  if (delaySec) {
    task.scheduleTime = {
      seconds: Number(delaySec) + Date.now() / 1000,
    }
  }

The ability to set the headers for the call is really very nice, as it opens up a lot of functionality if you wanted to add in a Bearer token, for instance. But you'll have to be careful about the time... the same data will be used for retries, so you would have to give it sufficient time on the token to enable it to be used for any retry.

With this, I was able to put together the sequence of Tasks that would quickly dispatch the processing, and return the original webhook back to Stripe. Quite nice to have it all done by the Cloud Tasks... AWS would have required that I process the events off an SQS queue, and while I've done that, it's not as simple as firing off a Task and fogetting about it.

Nice tools. 🙂

I just got off the phone with a very good friend that helped me see something in a way that was always there, but I wasn't extending myself to see it from that point of view - and it really got me to thinking: How wrong was I really?... and the answer was: A lot. 🙂

We all live our little lives, and it's unusual to meet someone that can really see life from a few completely different perspectives. The most common way I know of is profound loss - someone recovering from the loss of a close loved one will have the ability to see life with that person, and without that person, and their perspectives will be entirely different. Grief changes most people. But that's not the only way people can have different perspectives.

I was talking to my friend, and mentioning that I was going through a tough time with some folks, and she suggested that I had it all wrong. And proceeded to tell me how wrong I was.

She pointed out that life really is what we make of it, and that I could choose to see things as how they effected me, or I could see it from a different perspective, and see that there were other ways of handling the exact same thing, and in a different way, not make it an us-vs-them situation.

I'm thankful for my friend, because it was what I needed to hear - even if I didn't want to. I need to change how I approach things... life doesn't have to be kind... there's no rule about that. But we can choose to insert kindness in what we do, and not let the kill-or-be-killed be the way we live our lives.

Sometimes I am really wrong. By a lot.

And I'm glad my friend was there to help me see it.

I'm looking at the stories this morning about the actions in Washington by the pro-Trump mob that breeched the Capital for the first time since The War of 1812... and I feel certain the outgoing administration doesn't care - we've seen four years of policies, speeches, actions, and inactions, that show all that are paying attention that a day like yesterday was virtually inevitable.

Sure, it could have been 4 years from now... but it was now, and we have seen it. And this morning, it's now clear that those that were objecting to the Electoral College vote count have changed their minds, and have decided that they would object no more. Enough was enough - finally... for them. And we have some sort of closure for this election.

We also have a 50-50 Senate, and I have to smile at the idea that the Founders expected this might happen, so they planned for it, and work will get done, compromises will be made, people will be unhappy, and some will be happy. Life will go on.

I wonder what will come in the days and weeks ahead... will we really have seen enough? Will we really change how we treat the less-well-off among us? I hope so. I really do.

A close friend, and his family, all have come down with COVID-19 as a result of some Christmas family time. I am so sorry for them. I know the intentions were good - and the precautions the family took were meant to keep everyone safe, but this is a nasty flu, and it's indiscriminate to whom it will hit. Closed spaces for several hours - like a Thanksgiving or Christmas celebration is just what it likes.

I think my household got it back in February 2020, after some trips to Seattle and San Francisco I had to take for work. I was in a packed airplane, and those cities at that time were hot spots. I got sick, for a few days, and then got better. My daughter was sick for a few weeks, and got better. While we haven't been tested, we survived without major complications.

I hope my friend's family does as well. This is a scary thing to have happen...

Another trip around the sun... I hope for just a nice, quiet day, and then Pizza tonight for dinner. It's my special day, so why not treat myself to pizza? 🙂

It's been a heck of a year, with so many things happening like never before... the pandemic, the election, the working at home, the rationing of paper towels... it's been a year of exceptional events. And not just for me... for everyone. But we keep going.

So I hope that everyone has a little breather today... a little rest... so that we can get back at it on Monday.

A few days ago, I sent an email to the iTerm2 developer, and asked him the following question:

...and maybe this is a silly request, but I would really enjoy the option to put the Emoji Picker on the TouchBar of my MacBook Pro while in iTerm2… I know it’s not a Big Deal - so dragging it off/on in the customization makes sense, but there are a lot of times in my Git commit messages that I’d like to be able to toss in an emoji

and this morning I got a (surprise) response:

Thanks for pointing this out. There’s no good reason why it shouldn’t be allowed. Commit 1ae34d90f adds it. You can test in the next nightly build, due out in about an hour. In order to avoid breaking existing setups, it’s not in the default setup. You need to choose View > Customize Touch Bar to add it.

which is perfect for what I was hoping to have.

One of the best uses I've found for the TouchBar on the MacBook Pro is the Emoji Picker - as it's perfect for Instant Messaging, and Twitterrific, and at The Shop it's a big thing to have a nice, representative emoji as the first character of a pull request title. This is OK with LaunchBar, but it's not as convenient as the TouchBar Emoji Picker, and that's really what I was hoping to use it for. But until recently, iTerm2 just didn't allow it in the configuration of the TouchBar.

I am as pleased as I can be. Sounds silly, but it's nice to see that your thoughts aren't completely left-field to others. 🙂

UPDATE: the v3.4.4beta2 release has the Emoji Picker. I'm just smiling. 🙂

At The Shop, we are using a completely different platform than I've used in the past - Node/JS and Express as well as Platter, for a back-end database. It's been a steep learning curve for me, but I have to say today was a nice day where I really started to feel like I was getting the handle on the tools. What has really exciting to me with Express is the ease with which I can build middleware to insert into the calling stack.

For ring middleware, in Clojure, it's not horrible, but it's not trivial to understand the calling order, and the passing of the handler to all the middleware. In Express, it's far simpler - you simply have a function that takes the request, the response, and the next in the line of the calling stack, and that's it. You can augment the request, and that's basically what a lot of middleware is about - adding authentication tokens, looking up permissions, etc. It's all adding to the request to be used in the simpler endpoint routes.

When working with Passport for the authentication framework, it's great that it fits in with Express so well, but one of the issues that I ran into today was that the middleware added to the top-level Express app would be executed before the Passport authentication middleware that was in place on each individual endpoint. It makes sense, not all endpoints need authentication, so adding that with Passport would naturally be done after the top-level middleware. But that makes some of the middleware I'd written unfunctional.

The Passport authentication scheme can be set up to easily add the user object to the Express request, and then for all endpoints, it's "Just There". I had expected to add middleware that would take that user and use it to look up other attributes and data to add to the request as well. But if the middleware I'd written was placed at the top-level, then it wouldn't have the user object on the request, and so it'd never work.

The solution was so elegant, I'm convinced that this had to be anticipated by the Express developers. 🙂 Each of the routes wired into the Express app takes a path and a router:

  app.use('/login', loginRouter)
  app.use('/company', companyRouter)

and when you add in the Passport support for JWT authentication with a Bearer token, you get something like:

  app.use('/login', loginRouter)
  app.use('/company', passport.authenticate('jwt', { session: false }), companyRouter)

where the /login endpoint is not protected by the JWT, and the /company endpoint is. This seemed like a very late stage to put in the Passport middleware, but as it turns out, Express can handle an array, or a list of middleware in the use() function. So we can say:

  const authStack = [
    passport.authenticate('jwt', { session: false }),
    tenantMiddleware,
    accountMiddleware,
  ]
  app.use('/login', loginRouter)
  app.use('/company', authStack, companyRouter)

where the authStack is the additional middleware for the individual routes, and it's handled in the order it appears in the array.

And it works like a champ. Just amazing, that we can create different stacks of middleware and then as long as we layer them properly, we can set up an amazing diversity of middleware. For this, it's great that we can group the authentication-focused middleware into an array, and then easily drop that on the endpoints that need it.

Very slick. 🙂

Yesterday started Advent of Code 2020, and it looks to be a fun theme - Vacation - and while I'm sure it's been planned well in advance of the current pandemic, I'd like to think that the folks in charge are thinking of all of us, and making it a little more enjoyable this year... because we've been cooped up all year.

The puzzels start easy, and then build through the month, and that's just as it should be - an easy way to introduce folks to coding with fun problems as opposed to so many made-up teaching problems they run into. They always make me smile.

Is that just what we all need now? A smile? 🙂