What's it all about, Alfie?

It's been fun to get access to the beta of GitHub's Codespaces, but one of its short-falls is that when you run an outward-facing service - like a Jetty server, the IDE understands the port needs forwarding, but on the iPad, in Safari, there's really no way to forward a port. Additionally, the IP address of the container on the back-end isn't opened up for answering those forwarded requests. So while it's a great idea for development on an iPad - you can't really do any RESTful service development.

Additionally, the nice thing about it being all browser-based, is also a limitation in that it's browser-based, and no local storage. This means that there is no offline mode for editing, and while we can't (yet) compile on the iPad, you can edit - if the files are local, and without local storage, you don't have that.

So I went looking and found a very close match to what I might write: Buffer Editor. It's on iOS and iPadOS, and it allows for local and remote editing from an incredible number of sources - Dropbox, iCloud, GitHub, BitBucket, etc. For example, you can clone a GitHub repo right on your iPad, and then edit it locally, and push up changes. You can also set up an SSH/SFTP connection and remote edit the files, and even have a terminal session.

This is a lot like Panic's Code Editor for iOS, but Buffer Editor handles general Git access, and Code Editor does not. Also, Buffer Editor handles Clojure syntax, and Code Editor doesn't.

I was able to write to the Buffer Editor folks, and give them updated rules for Clojure, and within a week, they had an update out, and the changes were there. That's some impressive support. I have done the same with Panic, but I haven't heard back yet - there, I know they know Git support is important, so I'm thinking they may not be really supporting Code Editor on iOS as much... that would be a shame.

Still, Buffer Editor is working great - but I needed to have a host on the back-end to be able to do the work. I wasn't a huge fan of AWS, so I decided to try Linode, and I'm so very happy that I did! 🙂

Linode is a lot like AWS - with a somewhat limited feature set. You can get machines - either shared CPUs, or dedicated ones... and you can pick from a lot of different styles: compute, big memory, GPUs, etc. It's all most folks would need for most projects. They also have lots of SSD disk space - like NFS, and they also have an S3-like object storage. Add in load balancers, and it's enough to do most of the things you need - if you roll your own services like database, etc.

They also had a nice introductory offer so I decided to take it for a spin, and see what Buffer Editor could do with a nice Ubuntu 20.04 back-end with AdoptOpenJDK 11, and Clojure.

The basic instructions are very clear, and it was easy enough to set up the root account, and get the box running. It was even a little faster than AWS. Once I had the box running, I logged into the box, and updated everything on the box:

  (macbook) $ ssh root@123.45.67.88
  (linode) $ apt-get update && apt-get upgrade

With all that OK, I then set the hostname, updated the /etc/hosts file, made my user account, and got ready for moving my SSH key:

  (linode) $ vi /etc/hosts
             ... adding: 123.45.67.88   opus.myhome.com
  (linode) $ adduser drbob
  (linode) $ adduser drbob sudo

and then as me:

  (macbook) $ ssh drbob@123.45.67.88
  (linode) $ mkdir -p ~/.ssh && chmod -R 700 ~/.ssh

and then back on my laptop, I send the keys over:

  (macbook) $ scp ~/.ssh/id_rsa.pub drbob@123.45.67.88:~/.ssh/authorized_keys

And then I can add any options to the /etc/sudoers file as the command above put my new user in the sudo group, but there could be tweaks you might want to make there.

At this point, sudo was working on my account on the Linode box, and then it was time to lock down the machine a little:

  (linode) $ vi /etc/ssh/sshd_config
             ... PermitRootLogin no
                 PasswordAuthentication no
                 AddressFamily inet
  (linode) $ service ssh restart

At this point, I could install the other packages I needed:

  (linode) $ apt-get -y install --no-install-recommends openjdk-11-jdk leiningen grc mosh
  (linode) $ apt-get -y install --no-install-recomments postgresql postgresql-contrib

Then I can make a new user for Postgres with:

  (linode) $ sudo -u postgres createuser --superuser drbob

and then I can clone all the repos I need. The box is ready to go.

With this, I can now edit offline on my iPad, and then push, or copy the files when I get to a network connection, and then I can edit and debug as much as I'd like when I do have connection. It's almost exactly what I was hoping for.

The one missing thing: panes for the terminals... I'd like to have a REPL, and a tailing of the log file visible at the same time. I think I can accomplish that with the screen command, but I'll have to experiment with it a lot more to find out. But it's close... very close... 🙂

This morning I did a little looking for any tools that worked in the clj/deps system that was like the Leiningen plugins that did color syntax highlighting, and data formatting that I find very useful when the result of a function is a complicated data structure, or a sequence. I have been using whidbey as it works just fine with JDK 11 and any Clojure past 1.8.

The issue is that a good friend switched to the clj/deps package management and build tool for Clojure, and likes it quite a lot. So I've been trying to built up a way to do the same kinds of things in that ecosystem as with Leiningen. I didn't have a lot of luck the last time I looked, but maybe something was out there now? Or maybe I'd get a little lucky on the words I used in the search?

I found something that was close: rebel-readline. This is nice in that it's a simple readline replacement for the REPL, so it's not something that has to have a special environment - or a different app. I liked that it was able to do a lot of things on the readline that whidbey couldn't - because whidbey wasn't really active on the "read part" of the REPL - but more really on the "print part". This was nice, and I really liked that it was capable of showing the Clojure docs on the function - and it was fast, so that's a plus... but almost by definition - it wouldn't do anything on the output. So that wasn't as successful as I'd hoped.

Then a friend pointed out Reveal - and this was a lot more than I was looking for - the graphs, the processing of the output... this is a lot more like Gorilla REPL, that I've used, and written about in the past. It's all done in a browser, and Reveal is in a terminal session - so it's not exactly the same - but it's certainly something that is more of a joint "formatter and visualization tool" than just syntax highlighting and formatting of the REPL output.

I'll keep the links around, and maybe someone will write something like rebel-readline - but for the print (output) loop for the REPL... that would be nice. Until then, we carry on...

I've tried a couple of times to run a Jetty web server in a Clojure project on a GitHub Codespace, and the directions for port forwarding are very clear, and very easily done in the IDE, but when I try to "Open in Browser" - while in Safari on my iPad, I get nothing. And when I "Copy URL" I get 127.0.0.1:8080 which is "correct" as far as the documentation goes, but it's not working because in a browser, there's no way a browser page can redirect the localhost address.

So the only way this is going to work is if the VS Code IDE can open up a web page to the back-end, and show it there. Or... if they can open up ports based on the URL to get to the Codespace. They are all unique, so it's not impossible to image that... but it'd take work.

This is going to be important because as you develop things, you need to be able to interact with them, and being able to hit a web server, or a RESTful interface is kinda important. But hey... this is still in beta, and I noticed this morning that they have been making changes to the Codespaces page on GitHub, so I'll give them time.

This morning I wanted to see if I could get a more convenient method of getting to the attached Docker postgres server on GitHub Codespaces. The defaults of localhost, on port 5432 are standard, but if the machine can do unix sockets, then that's the preferred default. So there's the rub - the Codespaces images are Ubuntu, but they aren't set up for that - Docker maintaining the independent images and mounts for data. So I had to try something different.

I had previously tried using the pgpass file, but while that was easy to set up, it wasn't assuming precedence over the unix sockets... so that was a bust. The next thing I tried was to use the environment variables: PGHOST, PGPORT, PGUSER - and once I set those:

  $ export PGHOST=localhost
  $ export PGPORT=5432
  $ export PGUSER=postgres

then I could use:

  $ psql postgres

and other postgres commands like:

  $ createdb advent

to make a new database that I could reach with:

  $ psql advent

This is exactly what I was looking for! At this point, I could use the Codespaces for all the kind of development I was looking to do. Just fantastic! 🙂

I then updated the .bashrc and .zshrc files in my dotfiles repo so that any new Codespaces I would make have these baked in. I just need to get the Codespaces support directory into each project, and that will spin up just exactly what I need. Very nice indeed.

Today I noticed that a new beta of iTerm2 was out, and as part of the update, there is - of course - a restart of the app. But something I noticed, quite by accident, was that if you double-clicked on the tab title, iTerm2 would bring up a nice dialog box where you can enter the name of the tab, or even evaluate a function for the name of the tab.

In the past, I always had the ANSI escape codes to set the name to the current directory, and that was nice, but it also wasn't exactly what I wanted for a lot of my development work - because the tabs there need to have fixed names for the repo, or the function of the terminal, etc. So I had made a simple script functions:

  #
  # These are simple functions that can't be expressed as aliases, but
  # are very simple, and can go here because they are simple.
  #
  function winname() {
    echo -ne "ESC]0;$1^G"
  }
 
  function tabname() {
    echo -ne "ESC]1;$1^G"
  }
 
  function fixwt() {
    unset PROMPT_COMMAND
    if [ "$1" != "" ]; then
      winname "$1"
    fi
  }

so that I could easily override the PROMPT_COMMAND setting of the cwd, and the title would be fixed. It worked, but it meant that every time I had to restart iTerm2, I had to update all the shells with a fixit name, where name was the fixed name I wanted on that terminal tab.

But with this new iTerm2 feature, I don't have to do that.

I can simply set each one with the title I want, and leave those blank that will default to the PROMPT_COMMAND setting, and then they survive restarts! Amazing. 🙂

Yes, it's not really all that shocking, but for many years, I've hoped to have this feature, and now it's here, and I can restart my iTerm2 app, and not have to spend the next several minutes typing the same fixwt titles over and over. It's very nice.

Now if iTerm2 could remember the Spaces the windows were on... now that would be really nice! 🙂

This morning I was able to make a stripped-down dotfiles GitHub repo that Codespaces will use when a new Docker image is created. If there is a install file in the repo, it'll be executed after cloning the repo to the new user - as specified in the Dockerfile's USER variable.

So in my Dockerfile I now have:

  # Make my user for when we really get going, and pull in my env from GitHub
  ARG USERNAME=drbob
  ARG USER_UID=1001
  ARG USER_GID=$USER_UID
 
  # Create the user and add him to the sudoers so that he can install more stuff
  RUN groupadd --gid $USER_GID $USERNAME \
      && useradd --uid $USER_UID --gid $USER_GID -m $USERNAME \
      && apt-get update \
      && apt-get install -y sudo \
      && echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME \
      && chmod 0440 /etc/sudoers.d/$USERNAME
 
  # [Optional] Set the default user. Omit if you want to keep the default as root.
  USER $USERNAME

and that lands me in the repo's home directory, but I still have to clean up the dotfiles repo, and all the permissions from the last post. To do that, I made a simple function in my ~/.bashrc that does all that:

  #
  # This is a simple function to cleanup the GitHub Codespace once it's been
  # created. Basically, I need to remove the left-overs of the dotfiles setup
  # and clean up the permissions on all the files.
  #
  function cleanup () {
    pushd $HOME
    echo "cleaning up the dotfiles..."
    rm -rf dotfiles install README.md
    echo "resetting the ownership of the workspace..."
    sudo chown -R drbob:drbob workspace
    echo "cleaning up the permissions on the workspace..."
    sudo chmod -R g-w workspace
    sudo chmod -R o-w workspace
    sudo setfacl -R -bn workspace
    echo "done"
    popd
  }

At this point, I can launch a Codespace, it'll have all my necessary environment, and I can continue to refine it in my dotfiles repo as that's only being used for the set-up of these Codespaces. So it's easy to remove things I'll never need there, and make sure it's customized to the things that I do need there.

Very nice. 🙂

Late last week, I was accepted into the GitHub Codespaces beta program, and since I've been looking for something to do coding on my iPadPro, and was doing a little work with GitPod, it was something I was really interested in seeing and comparing with GitPod. So I started with two of the projects I've put into GitPod: Advent of Code, and CQ.

First things, Codespaces is a similar in that it's all about Docker images of the repos with enough of a system wrapped around it to enable the style of development needed. Java, Ruby, Node, Rust - if you can do it on Debian or Ubuntu, then you can do it in Codespaces. It's just that universal.

It's also possible to run other images, like Postgres, Mongo, MySQL, and then have them linked up to the repo's instance so that you can refer to the databases off of localhost and the default port. That's really nice.

It's not particularly easy - unless you really understand Dockerfiles... but if you keep at it, they have examples, and for the most part, you'll be able to get something going, and it's all in the repo, so it's simple enough to drop the image, and create another. It just takes time.

What I did find interesting is that the default user for Codespaces is root. I was not at all interested in working as root - even on a Docker image... it's just too uncomfortable. Thankfully, there is a vscode user you can use, and the USER directive in the Dockerfile will drop you into that user when things get started.

The next thing is the permissions... it's wild... everything is open to the world, and again, while this may be "fine" - it's very uncomfortable for me, so I converted the ownership of the workspace to the vscode user, and then removed the additional ACL security with:

  $ cd /home/vscode
  $ sudo chown -R vscode:vscode workspace
  $ sudo chmod -R g-w workspace
  $ sudo chmod -R o-w workspace
  $ sudo setfacl -R -bn workspace

With these done, the directory is owned by vscode and it's respecting the normal umask values on the creation of new files.

In the future, I'm going to have to figure out how to personalize the codespace by using the dotfiles repo in GitHub, and then installing my dot files and updating all these permissions, etc. as well. If I get to have several of these, it'll pay off to have it all done properly from the start...

Just saw a tweet from the AdoptOpenJDK folks, about new releases for JDK 1.8, 11, and 14, and thought I'd update what I had so that I could be assured of being able to upgrade when a serious bug shows up. It's good to be prepared. 🙂

Because I have several versions installed, I needed to upgrade each... and because they are delivered as casks, the upgrade commands are just a little different:

$ brew cask upgrade adoptopenjdk8    ;; 8u262
$ brew cask upgrade adoptopenjdk11   ;; 11.0.8
$ brew cask upgrade adoptopenjdk     ;; 14.0.8

Each one will take a bit to download, and they clean up the older version of the same package - so you only have the latest... not bad, and I can see the advantages.

All up to date now! 🙂

I've been a big fan of Repl.it as it allows me to be able to fire up a nice Clojure REPL without a lot of grief or overhead, and it's fast enough for small projects, and while it's not perfect - like you can't include a real project.clj so you can't load other packages, it's still pretty nice.

A few weeks ago, I noticed an odd little bug in the editor on Repl.it - the cursur wasn't where the actual insertion point was on the editor:

The more you had on a line, the more of a gap there would be on the editor. And it didn't matter if I was using the Desktop browser on my iPad, or the Mobile browser... on my iPad, it was off. And I tried a lot of things... reported it to the Bugs List for Repl.it, and while others had seen it - there were no answers.

Finally, I thought about the zoom feature.

On my iPadPro, for Repl.it, I like to zoom out a few steps to get more on the screen. I don't mind the smaller fonts - I can read them just fine, and it reduces the "dead space" on the screen quite nicely, so that I have a good editor window, and a nice REPL window.

So I went back to Repl.it, pulled up a saved REPL, and rest the zoom to "Original". Boom! The cursor and the insertion point lined up, and looked just fine. I then updated my Bug Report on Repl.it, and hoped that it was going to be a lot easier to reproduce for the developers - because I had a way to make it "Good", and then "Bad", and back to "Good". Repeatable 100% of the time!

It's been a few weeks, and nothing, so today I offered to help work on this, as I'd really like to have this fixed, and I'm sure others would too... but I may have to wait for iPadOS 14, and hope that Safari on iPadOS 14 is going to fix this behavior.

I'd be happy to help... because I'd really like it fixed before the Fall.

It's time to make sure that my laptop has the latest version of PostgreSQL, as I was reading that Heroku was moving to Postgres 12 as their default database. And since I don't do it that often, I decided to write it all down so that it's easier to do next time.

The process isn't bad... dump all the databases into one file, stop the server, update Homebrew, update postgres. This gets us to the point that we are ready to rebuild the new database:

  $ pg_dumpall > dump.sql
  $ brew services stop postgresql
  $ brew update
  $ brew upgrade postgres

Now we need to move out the old database data, create a new structure, and restart the service:

  $ cd /usr/local/var
  $ mv postgres postgres.old
  $ initdb -D /usr/local/var/postgres
  $ brew services start postgresql

You then need to go back to the directory of the first command - the one where you dumped the databases, and reload them all:

  $ psql -d postgres -f dump.sql

and at this point, everything should be back and running:

  $ psql --version
  psql (PostgreSQL) 12.1
  $ psql -l
                                      List of databases
      Name     | Owner | Encoding  |    Collate     |     Ctype      |  Access privileges
  -------------+-------+-----------+----------------+----------------+---------------------
   health      | drbob | SQL_ASCII | en_US.US-ASCII | en_US.US-ASCII |
   inventory   | drbob | SQL_ASCII | en_US.US-ASCII | en_US.US-ASCII |
   northhollow | drbob | SQL_ASCII | en_US.US-ASCII | en_US.US-ASCII |
   postgres    | drbob | SQL_ASCII | en_US.US-ASCII | en_US.US-ASCII |
   template0   | drbob | SQL_ASCII | en_US.US-ASCII | en_US.US-ASCII | =c/drbob           +
               |       |           |                |                | drbob=CTc/drbob
   template1   | drbob | SQL_ASCII | en_US.US-ASCII | en_US.US-ASCII | drbob=CTc/drbob    +
               |       |           |                |                | =c/drbob           +
               |       |           |                |                | _postgres=CTc/drbob+
               |       |           |                |                | postgres=CTc/drbob
   test        | drbob | SQL_ASCII | en_US.US-ASCII | en_US.US-ASCII |
  (7 rows)

At this point you can remove the old data:

  $ rm -rf /usr/local/var/postgres.old
  $ rm dump.sql

and everything is updated. You can then use any of the normal tools, including the Apache/PHP/Postgres that Apple makes available, and Postico - a great GUI client.

UPDATE: a friend asked: "What's in 12?" and I had to say I really didn't know... so I looked it up. The big changes are all about under-the-hood performance, and so, as I suspected, it just "works better". Which is really the goal, right? 🙂

I know I will probably enjoy the JIT on by default, as there are plenty of times stored procedures are in my databases, and that makes them faster, which is always nice. But it's just a great database - getting greater.