What's it all about, Alfie?

Interesting, that a few days after I move off Oracle's JDKs for my laptop, RedHat announces that it's going to assume stewardship of OpenJDK 8 and 11 from Oracle. It seems that Oracle wants to get out of the enterprise java business as it moved J2EE support to the Eclipse Foundation, and this just makes it clear that they really don't want to be in the Java business - at all.

This could be that they can't make enough money on it - or that they are just more interested in GraalVM, which they control, and can be used to compile a large subset of Java code into machine code. With this control, what do they need with the OpenJDK spec? It's just weight and cost. Better to get someone else to take care of that, and get back to making money licensing what they can.

It's just sad, but all too familiar, with Oracle. Glad I moved off. I'll stick with AdoptOpenJDK until there's a reason to move.

This morning I was looking back at an older project I had on macOS - Potentials, an engine that solves the 2D Poisson equation for a space where you can place conductors, dielectrics, and charge sheets, all of different shapes and sizes, and then use the vecLib CLAPACK call to solve the system of equations very quickly. Finally, it outputs the V(x,y) and E(x,y) to a file for plotting.

I remember writing it on a NeXTSTEP 3.3 Turbo Color Slab, because there was a plotting package that could read the data file, and until I could get around to putting that code into the app, I'd be able to check the results on the plotting package - HippoDraw. Sadly, that app moved to what it is today, on Windows, and I was just not motivated enough to do the plotting on NeXTSTEP.

This morning, I decided to get it out, and get it going on macOS 10.14.4 and Xcode 10.2. I had expected some challenges, and I was not disappointed, but I was also not denied.

The first thing was to allow Xcode 10.2 to tell me of the issues. Some were easy localization fixes, others weren't as easy to see. For example, vecLib has been merged into the Accelerate Framework. So in the code, where I'd had:

  #import <vecLib/clapack.h>

I had to replace it with:

  #import <Accelerate/Accelerate.h>

And then I dropped the vecLib Framework from the list of Frameworks to link to - face it - it was gone. 🙂 Then I needed to add in the Accelerate Framework to resolve the CLAPACK functions. This was something that I had forgotten to do, but it's really not too bad.

At this point, it built and ran just fine.

I then removed the old origin from the git repo, created a new repo on GitHub, and then pushed it all up there. It's so much more convenient to have it up there, as opposed the old git server that I had running on my own network. Just not even close.

Finish it off with a decent README, and we are now ready to work on writing that visualization of the output that I've wanted. Because it seems that there is no really good open source ObjC graphing packages for flattened 3D plots into 2D, and that's what I need - something that can plot these electric fields in 2D. Something to work on going forward.

This morning I decided to see if I couldn't fix the Swift highlighting on this blog by updating to the latest GeSHi 1.0.9.0 - which is now using PHP 7, and as it turns out, so is HostMonster, so I'm in luck! 🙂 At the same time, I really wanted to document all the components and links so that this post makes it a lot easier to update things later.

As a point of reference, the CodeHighlighterPlus project is based off the CodeHighlighter WordPress plugin, and it's not bad - it's just not been updated in many years, and there are a lot of added languages in that time. Still... I tip my hat to the original authors, as I couldn't have done it without them.

The steps to do this required a little digging, but that wasn't bad - in that I had a few posts about this process already, and so let's just repeat them here to make sure it's all clear for the next time. I started by making sure that my local CodeHighlighterPlus plugin was up-to-date with the GitHub repo. All good, so let's get the latest code from GeSHi, and just overlay it on the local repo. Replace where necessary, and then we're up to date with GeSHi... but it's not really cleaned up the way I like it.

The next thing was to update the geshi.php file for a few changes. The first thing I wanted to tackle with CodeHighlighterPlus was the line numbers. There was far too much space between the lines in a code sample with line numbers. This is corrected simply in the style for the lines:

    /**
     * Line number styles
     * @var string
     */
    protected $line_style1 = 'font-weight: normal; vertical-align:top;';
 
    /**
     * Line number styles for fancy lines
     * @var string
     */
    protected $line_style2 = 'font-weight: bold; vertical-align:top;';

to:

    /**
     * Line number styles
     * @var string
     */
    protected $line_style1 = 'margin: 0; font-weight: normal; vertical-align:top;';
 
    /**
     * Line number styles for fancy lines
     * @var string
     */
    protected $line_style2 = 'margin: 0; font-weight: bold; vertical-align:top;';

The last change is for the blank lines that start, and end, the code section when you use line numbers. It's just plain annoying. Change:

        // Get code into lines
        /** NOTE: memorypeak #2 */
        $code = explode("\n", $parsed_code);
        $parsed_code = $this->header();

to:

        // Get code into lines
        /** NOTE: memorypeak #2 */
        $code = explode("\n", $parsed_code);
        // remove a blank first and last line
        if ('' == trim($code[count($code) - 1])) {
            unset($code[count($code) - 1]);
            $code = array_values($code);
        }
        if ('' == trim($code[0])) {
            unset($code[0]);
            $code = array_values($code);
        }
        $parsed_code = $this->header();

and we are good to go with the changes to the code. Check everything in, push it up to GitHub and we're ready to deploy it.

At this point, I just need to deploy this to each of the WordPress sites on my server - and that's as simple as:

  $ cd public_html/blog/wp-content/plugins/CodeHighlighterPlus
  $ git pull

where blog is the directory in the WordPress content for the specific blog I'm working with. I simply use GitHub as the mechanism of deployment - with a pull system to make sure it doesn't mess too much stuff up.

In this release of GeSHi, we now have support for the following languages:

4cs            e              magiksf        qml
6502acme       ecmascript     make           racket
6502kickass    eiffel         mapbasic       rails
6502tasm       email          mathematica    rbs
68000devpac    epc            matlab         rebol
abap           erlang         mercury        reg
actionscript   euphoria       metapost       rexx
actionscript3  ezt            mirc           robots
ada            f1             mk-61          rpmspec
aimms          falcon         mmix           rsplus
algol68        fo             modula2        ruby
apache         fortran        modula3        rust
applescript    freebasic      mpasm          sas
apt_sources    freeswitch     mxml           sass
arm            fsharp         mysql          scala
asm            gambas         nagios         scheme
asp            gdb            netrexx        scilab
asymptote      genero         newlisp        scl
autoconf       genie          nginx          sdlbasic
autohotkey     gettext        nimrod         smalltalk
autoit         glsl           nsis           smarty
avisynth       gml            oberon2        spark
awk            gnuplot        objc           sparql
bascomavr      go             objeck         sql
bash           groovy         ocaml-brief    standardml
basic4gl       gwbasic        ocaml          stonescript
batch          haskell        octave         swift
bf             haxe           oobas          systemverilog
biblatex       hicest         oorexx         tcl
bibtex         hq9plus        oracle11       tclegg
blitzbasic     html4strict    oracle8        teraterm
bnf            html5          oxygene        texgraph
boo            icon           oz             text
c              idl            parasail       thinbasic
c_loadrunner   ini            parigp         tsql
c_mac          inno           pascal         twig
c_winapi       intercal       pcre           typoscript
caddcl         io             per            unicon
cadlisp        ispfpanel      perl           upc
ceylon         j              perl6          urbi
cfdg           java           pf             uscript
cfm            java5          phix           vala
chaiscript     javascript     php-brief      vb
chapel         jcl            php            vbnet
cil            jquery         pic16          vbscript
clojure        julia          pike           vedit
cmake          julia.bak      pixelbender    verilog
cobol          kixtart        pli            vhdl
coffeescript   klonec         plsql          vim
cpp-qt         klonecpp       postgresql     visualfoxpro
cpp-winapi     kotlin         postscript     visualprolog
cpp            latex          povray         whitespace
csharp         lb             powerbuilder   whois
css            ldif           powershell     winbatch
cuesheet       lisp           proftpd        xbasic
d              llvm           progress       xml
dart           locobasic      prolog         xojo
dcl            logtalk        properties     xorg_conf
dcpu16         lolcode        providex       xpp
dcs            lotusformulas  purebasic      xyscript
delphi         lotusscript    pycon          yaml
diff           lscript        pys60          z80
div            lsl2           python         zxbasic
dos            lua            q
dot            m68k           qbasic

And with the addition of Swift, I have highlighting on all the code snippets I've been adding. Very nice!

A couple of days ago, I found out about the licensing rules for JDK 11 from Oracle, and wanted to install OpenJDK 11 with Homebrew. Sadly, Homebrew doesn't really like the idea of installing older versions of packages, so I'm going to live with JDK 1.8, and OpenJDK 11. But that's not what this is about. 🙂 This is about Clojure, and Leiningen's REPL, and the fact that in that post I found a known problem with the current version of Ultra, and Leiningen versions greater than 2.7.1

Well... a little digging, and I find that the core component of Ultra is another project, and that same author has a similar REPL formatter and colorizer - called whidbey. Now it turns out that whidbey also has issues with Leiningen 2.9.0 - but at least it works with 2.8.3, so I can upgrade to 2.8.3, and then let them figure out the issue with nREPL 0.6.0 - as it's already an issue on whidbey.

So until then, I have both in my ~/.lein/profiles.clj, and I can flip between the two as needed:

  {
   :user {:plugins [[lein-exec "0.3.7"]
  ;                  [venantius/ultra "0.5.4"]
                    [mvxcvi/whidbey "2.0.0"]
                    ]
          :middleware [whidbey.plugin/repl-pprint]
          :whidbey {:color-scheme {:delimiter [:bold :yellow]
                                   :tag [:bold :yellow]
                                   :nil [:cyan]
                                   :boolean [:bold :cyan]
                                   :number [:bold :green]
                                   :string [:bold :red]
                                   :character [:cyan]
                                   :keyword [:yellow]
                                   :symbol [:bold :magenta]
                                   :function-symbol [:bold :blue]
                                   :class-delimiter [:blue]
                                   :class-name [:green]
                                   :exception [:bold :red]}}
          :ultra {:color-scheme ;; :solarized_dark
                   {:delimiter [:bold :yellow]
                    :tag [:bold :yellow]
                    :nil [:cyan]
                    :boolean [:bold :cyan]
                    :number [:bold :green]
                    :string [:bold :red]
                    :character [:cyan]
                    :keyword [:yellow]
                    :symbol [:bold :magenta]
                    :function-symbol [:bold :blue]
                    :class-delimiter [:blue]
                    :class-name [:green]
                    :exception [:bold :red]}}
          }
  }

At this point, I can switch to 2.8.3 with:

  $ brew switch leiningen 2.8.3
  Cleaning /usr/local/Cellar/leiningen/2.7.1
  Cleaning /usr/local/Cellar/leiningen/2.9.0
  Cleaning /usr/local/Cellar/leiningen/2.8.3
  3 links created for /usr/local/Cellar/leiningen/2.8.3

and I'm good to go with Leiningen 2.8.3, and as soon as whidbey is updated, I'll be able to move up to 2.9.0. Really glad I found this project.

[3/6] UPDATE: it looks like ultra is getting an update that will allow it to work with Leiningen 2.9.0 - which is great! I just need to wait for ultra 0.6.0 to hit Clojars, and I can update and use it. That would be very nice!

Today I've been having an email exchange with an old co-worker about an Open Source project that he's been working on - actually, it's pretty much done. It's all in Python, so it's not something that I'm really keen to use, but the design is good, and I can remember using something like it back at a previous job where we had to solve complex data modeling issues in as near real-time as possible.

The project is called dataflow and is really a compute/modeling system where you define Nodes and these nodes can be simple calculations on inputs, but they can be quite advanced - say rolling time averages, etc. and the key is that these Nodes can relate to one another in a simple Graph, that the user can describe, and then when any Node is updated, it fires off the updates of the dependent Nodes.

The nice thing is that these Nodes have a very simple API, and so they can do just about anything - averages of inputs, calculations, mean-square calls... it's really what the user wants to see. Tie this with a UI that displays the data, and you get something that is very powerful indeed. It can be a a Risk system where each Node can aggregate the child Nodes so that you can look across account, symbol, etc. and all updates as the input data updates.

It's been a nice reminder of some of the work I'd done at that job - but the work I did was in Java, and it was a bit more limited, but it still enabled some pretty nice apps to be built. Good stuff.

Well... today has been a very eventful day. I had noticed that Oracle dropped support for JDK 10, and mentioned it to some friends in Slack. I didn't know the details, so I decided to dig in and see if I could see anything on the net about it. I did. Lots. Changed my day.

Turns out, Oracle is changing it's licensing for JDK 11 - it's free for development and testing, but if you use it in production, it requires a license. A paid license. Yikes!

OpenJDK is also built by Oracle, and IBM, and RedHat - and it is Open Source - free for any use-case. So that's where people appear to be going. I can see that, but I've stayed away from OpenJDK because it's just the official JDK from Oracle. But now it seems Oracle is looking to cash in on the JDK, and this is just a lovely predicament.

But there is hope - kinda. Homebrew is supporting OpenJDK as a cask so that it can be versioned, and supported within the Homebrew ecosystem. That's good news. To install the latest, you simply need to:

  $ brew cask install java

and it will install it into:

  $ brew cask info java
  java: 11.0.2,9
  https://jdk.java.net/
  /usr/local/Caskroom/java/11.0.2,9 (64B)
  From: https://github.com/Homebrew/homebrew-cask/blob/master/Casks/java.rb
  ==> Name
  OpenJDK Java Development Kit
  ==> Artifacts
  jdk-11.0.2.jdk -> /Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk
  (Generic Artifact)

and the tools I built for switching JDK versions work just fine:

  $ setjdk 1.8; echo $JAVA_HOME
  /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home
  $ setjdk 11; echo $JAVA_HOME
  /Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home

That's the good news. What isn't really good is that there is an issue with the Java Collections where the toArray method added an additional arity and that breaks Clojure - badly. So it really doesn't help to have the OpenJDK 11 on the box - as Clojure can't make use of it.

However, that's not all... it appears that when I upgraded Leiningen, I broke one of the really nice plugins that I like to use with it - Ultra. The error is nasty as well:

  $ lein clean
  $ lein deps
  $ lein repl
  [WARNING] No nREPL middleware descriptor in metadata of
    #'clojure.tools.nrepl.middleware.render-values/render-values,
    see nrepl.middleware/set-descriptor!
  [WARNING] No nREPL middleware descriptor in metadata of
    #'clojure.tools.nrepl.middleware.render-values/render-values,
    see nrepl.middleware/set-descriptor!
  nREPL server started on port 54967 on host 127.0.0.1 - nrepl://127.0.0.1:54967
  ERROR: Unhandled REPL handler exception processing message
    {:id ad3092dd-8491-4d60-a1a9-5092d4d090a9, :op clone}
    java.lang.IllegalArgumentException: No implementation of method:
    :send of protocol: #'nrepl.transport/Transport

and if I removed Ultra from my ~/.lein/profiles.clj then everything worked fine again. But this was with Leiningen 2.9.0 - and that was an issue. In fact, it was an issue that's already been reported to Ultra. I added my notes, as the suggestion didn't work for me.

So now I'm left with downgrading Leiningen to get Clojure working again. This is not as easy as I'd hoped, but it's what needs to be done. First, get into where the Homebrew formulae are stored, :

  $ cd $(brew --prefix)/Homebrew/Library/Taps/homebrew/homebrew-core/Formula
  $ git log --follow leiningen.rb
  commit 2be19f3787d811247095e704765fc33a8815d639
  Author: BrewTestBot <homebrew-test-bot@lists.sfconservancy.org>
  Date:   Tue Feb 12 13:07:29 2019 +0000
 
      leiningen: update 2.9.0 bottle.
 
  commit 9ca7899818eb02cc8252d47cc32b5a42554a7655
  Author: Rahul De <rahul080327@gmail.com>
  Date:   Tue Feb 12 10:43:59 2019 +0100
 
      leiningen 2.9.0
 
      Closes #36927.
 
      Signed-off-by: Chongyu Zhu <i@lembacon.com>
 
  commit fd1c0ba8f162a3c41debef6143dca8ba19227f90
  Author: BrewTestBot <homebrew-test-bot@lists.sfconservancy.org>
  Date:   Sat Dec 15 10:45:41 2018 +0000
 
      leiningen: update 2.8.3 bottle.
 
  commit 40553be0b01710db9e16cff978f593b83ecdfe3b
  Author: Rahul De <rahul080327@gmail.com>
  Date:   Sat Dec 15 11:27:08 2018 +0100
 
      Leiningen 2.8.3
 
      Closes #35151.

at this point we heed the git SHA for the version we want to move to. In the case of Leiningen, the SHA for 2.8.3 is fd1c0ba8f162a3c41debef6143dca8ba19227f90 and the SHA for 2.7.1 is cdddd1094b26d0092e0030051dac97a286bb3fc4. I started with 2.8.3, and that didn't work, so I had to go back to 2.7.1. This is a big difference.

Then make a branch for that version, and reinstall Leininden from the Ruby file:

  $ git checkout -b leiningen-2.8.3 fd1c0ba8
  $ brew reinstall ./leiningen.rb
  $ brew switch leiningen 2.8.3
  $ git checkout master

for 2.8.3 and, for 2.7.1:

  $ git checkout -b leiningen-2.7.1 cdddd109
  $ brew reinstall ./leiningen.rb
  $ brew switch leiningen 2.7.1
  $ git checkout master

At this point, we can see that both versions are installed:

  $ brew info leiningen
  leiningen: stable 2.7.1 (bottled), HEAD
  Build tool for Clojure
  https://github.com/technomancy/leiningen
  /usr/local/Cellar/leiningen/2.7.1 (9 files, 14.7MB) *
    Poured from bottle on 2019-02-18 at 13:29:34
  /usr/local/Cellar/leiningen/2.8.3 (9 files, 13MB)
    Poured from bottle on 2019-02-18 at 13:22:14

Now it's possible to run Leiningen and Ultra - again. It has to be JDK 8, and Leiningen 2.7.1... but until Ultra is fixed for the change in Leiningen, this is as far as I can go.

What was even more annoying was that my new work laptop has a very recent version of Homebrew on it, and the git repo has been trimmed, and so there is no git SHAs for the previous versions. However, with tar and copying the tarball over, I was able to get the 2.7.1 and 2.8.3 versions there, and things are now working OK.

Wow! What a mess to get back to where I thought I started the day!

This morning I was working to upgrade my laptop from Postgres 10.3 to 11.1, and I did the following, as part of the upgrade:

  $ brew update
  fatal: could not read Username for 'https://github.com': terminal
  prompts disabled
  Error: Fetching /usr/local/Homebrew/Library/Taps/homebrew
  /homebrew-boneyard failed!

I did a lot of googling, and came up with a couple of ideas. This was from GitHub:

  $ git config --global credential.helper osxkeychain

but that didn't work. Then I found another reference saying that the 2FA on GitHub was the problem, and that you needed to bypass that with:

  $ git config --global --add url."git@github.com:".insteadOf "https://github.com/"

and still that didn't work. So I took a different approach, I started looking at the Homebrew files on my laptop - specifically, the boneyard mentioned in the error. Turns out... if you looked at the .git/config file in that directory, it points to a directory in GitHub that no longer exists! OK... this might be something.

So I removed the nearly empty directory, and tried again:

  $ brew update
  Already up-to-date.

Success! It wasn't the config of git - it was that Homebrew was trying to update something that no longer existed. Lesson learned.

It's time to make sure that my laptop has the latest version of PostgreSQL, as I was reading that they were making big strides in the windowing functions in postgreSQL 11. And with Homebrew 2.0 out, it made sense to just write it all down again, just so that I have it for reference.

The process isn't bad... dump all the databases into one file, stop the server, update Homebrew, update postgres. This gets us to the point that we are ready to rebuild the new database:

  $ pg_dumpall > dump.sql
  $ brew services stop postgresql
  $ brew update
  $ brew upgrade postgres

Now we need to move out the old database data, create a new structure, and restart the service:

  $ cd /usr/local/var
  $ mv postgres postgres.old
  $ initdb -D /usr/local/var/postgres
  $ brew services start postgresql

You then need to go back to the directory of the first command - the one where you dumped the databases, and reload them all:

  $ psql -d postgres -f dump.sql

and at this point, everything should be back and running:

  $ psql --version
  psql (PostgreSQL) 11.1
  $ psql -l
                                      List of databases
      Name     | Owner | Encoding  |    Collate     |     Ctype      |  Access privileges
  -------------+-------+-----------+----------------+----------------+---------------------
   health      | drbob | SQL_ASCII | en_US.US-ASCII | en_US.US-ASCII |
   inventory   | drbob | SQL_ASCII | en_US.US-ASCII | en_US.US-ASCII |
   northhollow | drbob | SQL_ASCII | en_US.US-ASCII | en_US.US-ASCII |
   postgres    | drbob | SQL_ASCII | en_US.US-ASCII | en_US.US-ASCII |
   template0   | drbob | SQL_ASCII | en_US.US-ASCII | en_US.US-ASCII | =c/drbob           +
               |       |           |                |                | drbob=CTc/drbob
   template1   | drbob | SQL_ASCII | en_US.US-ASCII | en_US.US-ASCII | drbob=CTc/drbob    +
               |       |           |                |                | =c/drbob           +
               |       |           |                |                | _postgres=CTc/drbob+
               |       |           |                |                | postgres=CTc/drbob
   test        | drbob | SQL_ASCII | en_US.US-ASCII | en_US.US-ASCII |
  (7 rows)

At this point you can remove the old data:

  $ rm -rf /usr/local/var/postgres.old
  $ rm dump.sql

and everything is updated. You can then use any of the normal tools, including the Apache/PHP/Postgres that Apple makes available, and Postico - a great GUI client.

I've been rebuilding a new MacBook Pro laptop from work because the security tools we use at The Shop don't allow Migration Assistant to work - which is sad, because it works so well under normal circumstances, but water under the bridge, and I have to reinstall a lot of things on the box - as if from scratch.

Because of Clojure, one of the things I have to make sure to install is the latest JDK. Actually, I need to install all the latest JDKs - 1.8, 9, 10, and 11. And then I have a simple little tool for changing the JDK version that I want to be using. Simple. Easy.

When I last went to upgrade the versions of the JDKs, I found that Oracle has dropped support for JDK 9. I had heard a lot of talk about how JDK 9 wasn't really doing well because of some of the design decisions... and so this wasn't really a surprise, but it was unexpected to see that Oracle had pulled it from the Downloads. But OK... they didn't want people using it, so they pulled it.

Today, I saw that Oracle had pulled JDK 10 as well. Now that was a bit of a surprise. I know that it wasn't as popular with the Clojure community as JDK 1.8, but to see a second abandoned JDK version... well... that's a heck of a realization from Oracle. It's not good... it's causing us too much to support it... drop it.

So I have JDK 1.8 and 11 on the new laptop, even though on my personal machine I still have the last available JDK 10. But at some point, that will have to be thinned out because it just won't be supported. Wild.

Hats off to the JDK 1.8 guys - You've made a heck of a product.

This morning I saw a tweet that announced that the Cognitect AWS Client library 0.8.243 was released. This was news to me, because I've been using Amazonica for years now, and it works, but has plenty of little things that always made me write a wrapper around all the things I really needed to do.

But hey... Cognitect are the guys that own Clojure - so if there's a "preferred way" to integrate with AWS, these guys would be the ones that would do it in the best way possible. I haven't looked at the docs much, but I wanted to put all this in a post so I could come back to it the next time I needed to talk to AWS from Clojure, and give this a spin.

Quite interesting...

UPDATE: I've taken a look at it, and I find it very interesting. Rather than use the AWS Java SDK, and just wrap the Java objects in Clojure code - which isn't bad but it doesn't really make it discoverable, and you end up wondering if there isn't a simpler way to get the data. With the Cognitect scheme, you send data, and get data, and it's up to you to then read what you need. Much nicer.