What's it all about, Alfie?

I was chatting with a Clojure developer friend this morning, and he mentioned getting the CryptoQuip solver running on the GraalVM and seeing how it performed. I didn't know anything about it, and I decided that if my friend was interested in it, then there's certainly something there, and I should read up on this. I'm very glad I did.

I've seen several attempts at making a Clojure compiler in the past - heck, I've thought about trying it - but each one falls a little short, and only handles a small subset of the complete Java byte codes, and so it's good for little things, but it doesn't really handle big projects that you'd run into in practice. So I started reading, and there is the GraalVM, and the Graal compiler. It's a project from Oracle, and is really about creating a universal VM that can run all kinds of languages - not just JVM-based, which is very interesting, and more than a little ambitious, but more power to them.

They have pages on compiling to native apps to avoid the startup costs of the JVM - which isn't horrible, but for Clojure, what's loaded is a lot more than just the JVM, and it really does make building command-line tools in Clojure hard. But I found another post about a real Clojure example, and it showed that it wasn't all that hard to add a post-processing step after the uberjar, and then you get a native executable that's pretty impressive.

When I did more reading, the big limitation is that code can't hit the class loader and be compiled - makes sense, but in my past one of the really nice libraries that used this to great effect was amazonica and it was really good at handling the AWS details, but I suppose if you're not using AWS, then you're not too bad off.

WHat's for sure is that I'll remember this as it makes Scala, Clojure, and all the JVM languages able to be native code, and that will make Docker images smaller, and everything spin up faster, etc. It even says it does Python, so there's a possibility there.

Very interesting.

After I went through Swift and Xcode, I was looking for something to read, and CloudKit seemed like a really good thing to read - as I know a friend is using it for his app, and it's one of the really nice features that Apple is making available for server-side uses in the iOS and macOS apps. I'm not at all sure when I'll use it, but I really wanted to get to know it so that I can know what it's capable of, and the basic limitations it's got.

To start, it's important to see this as what might be called a very simple database, and it's probably a good idea to take Apple's cue and call them RecordSets. It really is best to think of them as Collections, as it's not really a traditional database at all, but you can mimic a lot of database activity with it. Still - Collections is the best way to think about it.

Next, they have certainly put in a lot of work to make sure app developers have the tools they need to specify these records on the Apple developer web site. It's really impressive, but then again, Apple uses CloudKit a lot - so it makes sense that you make the tools that help you make apps.

What's interesting is that they have a very logical, clear, data access pattern that covers getting just the updated records... or sending a predicate and getting all the records that match that, or paging them as needed to make the app more responsive to the user. What I find very nice about CloudKit is that for thumbnails of the images - they don't do a thing - it's up to you, the app developer, to make those, and put those in the record. That way, you control the style, image quality, etc. of the thumbnail. Very nice.

I Like that they have the batch and single operation interface, and they have really gone all-in on the NSOperation as the tool for doing all this work. If you get used to that, and good at using it you're going to be set with CloudKit. They have subclassed that to have a lot of different operations, and true to NeXT and Apple form, the names are a little strange to me now - having done a lot of functional coding over the last few years - where names are typically a lot shorter, but that's just the way NeXT has always been.

As I was watching the WWDC sessions, and reading the docs, I realized that this could not really be made a lot simpler for the general case of interacting with iCloud, but it sure was a lot more complicated than any one app would really need. I kept thinking about how to simplify this and make it into a library with simpler access patterns. Then I'd realize that it would be restrictive, and so it would only be good for a subset of all use-cases, and again Apple did the right thing, and it's complex because it's a complex system.

Still... I can bet that if I had to use it in an app, I'd have to make a Framework or something to encapsulate the use cases that I was going to be using - just because I'd need to abstract this away to make it simpler to see how the part interacted with one another. But I suppose that another point to consider is that an app isn't about a lot of complexity, and so interacting with CloudKit is probably the most complex thing an app is doing, and so it's not really surprising that this is as complex as it is.

Low down: it's impressive, and full-featured, and I think it's great that Apple made it.

I read a tweet this morning about Apache Pulsar achieving top-level project status in the Apache organization. I hadn't heard of it, so I spent a little time reading about it and getting familiar with the differences between it and Storm, Flink, etc. What I found was something that I might actually use in the future.

The important parts are that Pulsar is a message broker with reliable messaging, with storage and fail-over. This makes it way better than many other systems, but it goes a lot further than that. Because it's Apache, it's using ZooKeeper, but it's also using BookKeeper for the message storage - which is an interesting twist.

It's scalable up to a million topics... is meant to sync servers by geo-location, and all the goodies you'd expect from something like Cassandra or the like. Lost of resilience. Turns out, it was the message broker for Yahoo, and has been open sourced. It's low-latency, and meant to be server-to-server in the datacenter, even if they are separated by continents. So it's battle-tested. And all this makes it a useful project because there are lots of times I can imagine needing a better broker than SQS.

But what makes it even nicer is that it has all the Lambda capabilities that AWS has as well. So now you can add in the streaming capabilities of Storm in the messaging system - as opposed to having a streaming system talk to Kafka as a source and sink. This is a different take on the problem, but one of the problems I had with Storm was the scaling of the nodes - it was hard to get them to not back-up. If this is a message broker, then we have a lot more flexibility on the processing time, and every hand-off is to a queue/topic. That's good news.

I'm not sure if/when I'll need to have something like this, but I'm very happy to see that they have it, and it's gotten to a point that it looks pretty stable. Good news!

Yesterday, around noon, I guess, macOS Mojave was released, and signified the end of a very exciting week for Apple. First it was iOS 12, and Xcode 10, then the new iPhones, then macOS Mojave. What a week!

I'm not going to use Dark Mode - it's just too hard for me to read the small text, but that's OK. I like that Apple is trying new things, and it's getting people excited about the UI again, which is always good. I do like a few of the UI changes that they added - there is now a clear distinction between the apps in the Dock, and those that are permanent members of the Dock. Nice. I also like that Safari 12 is getting more strict with the security and ad blocking. It's an entirely different world out there now - as opposed to 10 yrs ago, and I like that Safari, and Apple, feel that security and privacy is a fundamental human right.

There are tons of things under the covers, I'm sure. I've read about several, and as Xcode 10 was likely the build system, I'm guessing that there are nice performance boosts in several spots as well. All very cool stuff.

Late yesterday, a friend of mine was laid off. It happens, but he was on a new project at a new company... probably hadn't been there three months, and the project was cancelled. He was the only US-based employee for the company, and so that was it. No new project. No other opportunity in the company. Gone.

While a part of me understands this logic - you hire for a project, and if it is cancelled, then you don't need the people you hired for it, and so they can go. This makes perfect sense to me as a business owner. There is no fat. But at the same time, when the company has more than 50, or so, employees, and you find a good one, you have to wonder why they didn't plan for the possibility, and find a place for a good person to do good work?

I'll never know why they didn't, and how close they were to pulling the plug when he hired on, but it seems like the relationship between tech worker and employer is at an all-time low. The workers expect the world from the employer - tons of money, ping-pong, meals, stock, etc. and in return, they seem to want to work 4 hrs/day and be treated like royalty. Employers want to hire who they have to, and then drop them when they don't need them.

I suppose it's only fair, but it's sad at the same time. We're getting more like nature where the strong survive and thrive, and the weak, or unlucky, are eaten. Kinda harsh.

This morning I decided to take another look at the Xcode 10 performance tests on my CryptoQuip puzzle solver, but this time, looking at the Objective-C version and see if the Xcode 10 changes would make it any faster. Turns out, I'm very glad that I did.

I was getting a very odd error on the line:

int main(int argc, const char *argv[])
{
    return NSApplicationMain(argc, argv);
}

the value of argv was causing a SIGABRT - and I just had a feeling it was some setting in the old project file and I needed to clean those up - so after a little bit of updating, it compiled and ran just fine.

I then put in the timing of the call with a few lines:

- (BOOL) attemptWordBlockAttack
{
    // sort the puzzle pieces by the number of possible words they match
    [[self getPuzzlePieces] sortUsingSelector:@selector(comparePossibles:)];
    // ...now run through the standard block attack
    NSTimeInterval begin = [NSDate timeIntervalSinceReferenceDate];
    BOOL ans = [self doWordBlockAttackOnIndex:0 withLegend:[self getStartingLegend]];
    NSLog(@"%lu Solution(s) took %f msec",
        (unsigned long)[[self getSolutions] count],
        ([NSDate timeIntervalSinceReferenceDate] - begin) * 1000);
    return ans;
}

so that I'd get a similar result as the Swift version - basically, taking everything out of the time in the solution except the attack. That's as close as an apples-to-apples as I could get with the other versions.

What I found out blew me away:

CryptoQuip[87317:16056980] Loaded 25384 words from /Users/drbob/.../words
CryptoQuip[87317:16056980] Ready
CryptoQuip[87317:16056980] Solving puzzle: 'Fict O ncc bivteclnbklzn O lcpji ukl pt
                               vzglcddp' where b=t
CryptoQuip[87317:16056980] 1 Solution(s) took 0.464082 msec
CryptoQuip[87317:16056980] Solution found: 'When I see thunderstorms I reach for an
                               umbrella'

Now the times for Clojure were in the 6 msec... and Swift was about 7 msec... but I had to do a double-take on this... I had no idea Objective-C was going to be this fast! I had to compute, by hand, the number of milliseconds since the reference date - January 1, 2001 - just to make sure I was doing the math right. I was right... it was right... I checked the code several times, made sure I wasn't skipping the actual solution... and I wasn't.

Wow... Apple has really outdone themselves on this... sub-msec on the solution. Wow! I have to say this is excellent news in my book. I can live with Swift, and I know it's the future, and it's fast... but it's not this fast, and that's just amazingly cool!

Withe the recent improvements in performance from Xcode 10 for my Swift code, I thought I wanted to dust off DKit and make sure it still works on macOS 10.13.6 with Xcode 10 - it's been quite a few years since I've made any changes to it, and I just wanted to make sure it all builds and runs.

One of the things I remembered from a few months ago, when I tried to dust it off for a homework project for a job interview and I had some problems with the fact that the shared library was being built as a dylib and that just makes it a lot harder. At the time I originally wrote the library - some 15 yrs ago - there wasn't the support in Mac OS X for the standard unix SO shared library - so I had to use the dylib and figure out how to link it in properly.

Today, the story is entirely different - and honestly, for several years, I've known that macOS could generate the unix SO shared library. So I just switched over to that this morning, and gave it a spin. Turns out it worked perfectly.

I was able to compile all the components of the library, and then the test apps, where the templates are really being used, and all the tests ran. I checked in the simple changes to the Makefiles and everything is solid.

I'm sure there are a lot of things that this could add - I could make a test app that read a file and sent it out on a TCP or UDP transmitter - for the testing of the TCP and UDP receiver, but I'm not worried about that right now. It's enough that it's still working just fine, and builds on the latest macOS. Can't beat that!

This morning, with Xcode 10 updated yesterday with iOS 12, I thought I'd take a run at the Quipper code that is based on Swift 4.2. In the past tests on Xcode 9, I was seeing times in the 40 msec (Release build) and that was nice as it was fully 20 msec faster than ObjC on the same machine, but it's still a far cry from the 6 msec of Clojure on the same machine.

But I've heard a lot about the speed improvements in iOS 12, and it seemed like a smart thing to test the performance on the same code, on the same box - but with Xcode 10. So I did. Wow.

2018-09-18 06:59:04.76 Clearing out all old data for the attack...
2018-09-18 06:59:04.79 List of Possibles:
2018-09-18 06:59:04.79   ... bivteclnbklzn :: 1
2018-09-18 06:59:04.79   ... O :: 2
2018-09-18 06:59:04.79   ... ncc :: 29
2018-09-18 06:59:04.79   ... pt :: 40
2018-09-18 06:59:04.79   ... vzglcddp :: 46
2018-09-18 06:59:04.79   ... ukl :: 589
2018-09-18 06:59:04.79   ... Fict :: 1727
2018-09-18 06:59:04.79   ... lcpji :: 2168
2018-09-18 06:59:04.79 Starting the word block attack...
2018-09-18 06:59:04.79 working on word: 0 [bivteclnbklzn] - 1 possible matches
2018-09-18 06:59:04.80 working on word: 1 [O] - 2 possible matches
2018-09-18 06:59:04.80 working on word: 2 [ncc] - 29 possible matches
2018-09-18 06:59:04.80 working on word: 3 [pt] - 40 possible matches
2018-09-18 06:59:04.80 working on word: 4 [vzglcddp] - 46 possible matches
2018-09-18 06:59:04.80 working on word: 2 [ncc] - 29 possible matches
2018-09-18 06:59:04.80 working on word: 3 [pt] - 40 possible matches
2018-09-18 06:59:04.80 working on word: 4 [vzglcddp] - 46 possible matches
2018-09-18 06:59:04.80 working on word: 5 [ukl] - 589 possible matches
2018-09-18 06:59:04.80 working on word: 6 [Fict] - 1727 possible matches
2018-09-18 06:59:04.80 working on word: 7 [lcpji] - 2168 possible matches
2018-09-18 06:59:04.80 1 Solution(s) took 7.602294921875 msec

I was just shocked - 7.6 msec - that's within reasonable tolerance with the Clojure code - and all that improvement is in the compiler! I have to tip my hat to the Xcode engineers - that's an amazing improvement. I'm guessing that macOS Mojave is going to be quite a treat in a week, as it's been built with these tools. That's something very nice to look forward to.

For now, I'm super happy with the performance of Swift 4.2 - and while I'm not a huge fan of the syntax, it's got the performance that means it's something to take seriously, and use as opposed to Clojure which was problematic on the iOS and macOS apps anyway.

Great News!

Linus Torvalds is in the news again this morning, and I'm glad to see he's now aware of the effect he had on people, it got me thinking about git and I thought I remembered that he wrote the first version of git after something happened with the license on the SCM tool he was using on the Linux Kernel. So I hit Wikipedia, and sure enough, there was a license issue with BitKeeper, and so Linus wrote the first version of git.

What I hadn't remembered was the timeline - and that's what amazed me this morning:

• 3 April 2005 - Development Starts on git
• 6 April 2005 - the git project is announced
• 7 April 2005 - git is self-hosting
• 18 April 2005 - first merge of multiple branches
• 29 April 2005 - performance target is achieved: 6.7 patches per second
• 16 June 2005 - linux kernel 2.6.12 release is managed by git

So a couple of months to get a SCM that hosts one of the most influential codebases that currently exists... that's more than impressive... that's down right amazing.

Think about all that git has effected... GitHub, BitBucket, GitLabs - there's an industry built up around it - that's wild to think about. Sure, the guy may not be someone you want to have over for dinner, but what he accomplished in a few months was nothing short of amazing. And all of us in the tech community have benefited from it.

There is something to be said for talent... it's not necessarily sitting in the prettiest of packages, but we would all do well to look past the packaging, and try to see the real value under the covers. It can be revolutionary.

We are entering a very exciting week for Apple folks, such as myself. Today it's iOS 12 with the Memoji, automation with Siri, more efficient code to make a lot of things faster on older hardware, and soon - multi-person FaceTime. That last one is interesting to me as it really challenges Google Hangouts and GoToMeeting. They are good products, but this will be in all iPhones, and that's got a really big advantage in market-share.

I also read the comparison of the iPhone Xs Max with the 13" MacBook Pro - and the iPhone was certainly on par with the laptop... it's just crazy how much capability Apple is packing into these phones... I think I need to do a little looking into ARKit and see what it's got. I'm not really sure that I have an augmented reality idea in me, but it might be really nice to see how to do these things. You just never know.

Exciting week!