What's it all about, Alfie?

So I just got out of a meeting with my manager's manager. My manager is trying to rebuild an existing system - as opposed to simply augmenting it, and the way he's going about doing this is by using AWS services and a flock of 11 consultants. They are skipping what is, and jumping right to what he wants, and in so doing, leaving all the employees of the company that might offer help, support, and certainly ongoing maintenance - completely in the dark.

I've asked for details about the project I'm supposed to be on, and the response was always "None of your concern." OK... you're the manager. After repeated attempts to talk to HR, and seeing nothing done about it, I've simply taken the stand that I'll be very up-front about the fact that I'm doing nothing, as long as I'm given nothing to do. That way people can choose to see that as an anomaly - or not. Their choice.

I'm being paid either way. I'm very transparent with HR so that should it come back to me that I was wasting company resources (my time), that would be documented as not true. So I'm covered.

I don't think that using AWS is wrong. It's just a different approach than what I know the CTO has expressly stated: It just costs too much. It's more, but if you factor in the cost of having a datacenter, and racks, and machines, and people to maintain them, and then people to install, upgrade, monitor and fix the services like AWS offers, well... I'm not certain that all cases are clearly cheaper - one way or the other. I really think that the costs are coming down, and the service levels are to the point that you need to consider it on a case-by-case basis to be 100% certain.

But that's just my opinion, and not the CTO's. And he has made himself very clear on the matter. So for my manager (a consultant) to come in and make the plan, and hire the consultants to execute this plan that is 100% AWS... well... I knew it'd be a point of contention, and told my manager this when he first showed me his plans.

"Not your concern. That's something I'll make happen."

OK, boss... I tried to tell you. As my GrandPa Bowen said:

Those who don't listen, must feel.
-- Don Bowen

so he's going to have to feel it.

Part of that seems to be happening in the meeting I just got out of. The CTO said that AWS was not going to happen. That the enhancement of the existing system was the job, and that I was going to be brought in - as well as the other employees that have a solid understanding of the existing system. Makes good sense to me.

But it's 180 degrees from what my manager has been doing, planning, and moving towards.

So starting tomorrow I can see a new round in the fight. There's no way that the current plan is going to be allowed. So what value are the AWS consultants? My manager is being forced to bring in people he didn't want to bring in - and actively excluded and rejected. How hard are we going to work for this guy? Why should we? He was ready to drop us all - and told me as much less than a week ago. So how is he going to make this happen?

At the same time, the first round went to him in hiring the consultants, and starting this work... but this last round went to the CTO who shut down a $2 million deployment because we already had 90% of it in-house, paid for, and supported. It would be a colossal waste of money.

I'm no fool... this fight isn't over, just a temporary lull in the action. Soon there will be more slings and arrows, and we that are caught in the middle will no doubt soon tire of this, and let the (non-)Titans battle it out amongst themselves.

But is this any way to run a company?

This afternoon I saw a tweet from Heroku about them adding Redis to the add-ons for their service. This just a few days after their announcement that Postgres was available for the free tier, and the new "Free" tier for apps. They are getting aggressive with what services they are providing. This makes a ton of good sense to me, as I'm a huge fan of redis from clojure, and this makes all the experience I've got in building apps directly transferable.

While I know the Free tier isn't all that great, the idea that there is a Free tier is amazing, and it means that I can write something and throw it up there, and as it's needed, I can scale it up. Very cool. They also have a hobbyist tier that's only something like $8/mo. - similar to GitHub.

If I needed to be firing up a web service, it'd be clojure, redis, and postgres - all on Heroku. What an amazing service.

A really nice co-worker of mine has been doing some interesting work with random number generators for the test.check clojure library, and I decided this morning to have a look and see what it's all about. After all, I can't imagine something a lot better than the standard clojure.test as it's got everything I've ever needed. But Bret has used it, and Gary has to have started using it for a reason, so I wanted to find out what that was.

I'm very glad I did.

The problem that I've run into with simple unit test frameworks is that they allow you to test specific inputs and outputs. If you want something to run 100 times with random inputs, you could do it, but it wasn't easy. Certainly, the question as to Why? would arise unless you had a testing framework that thought not in terms of specific inputs and outputs - but in the structure of the data.

If we wanted to test a sorting function, then we could check to see if the act of operating on a sorted sequence was the same as the input sequence, then we'd need to be able to make arbitrary sequences, and then verify the structure of the data as well. This is then more like generators and properties - which is what test.check calls them.

In this regard, it's easy to see how you'd use this to test the limits of a function without worrying about making sure you got all the edge conditions in the unit tests. It's the shotgun approach, and in some cases, it can really make short work of finding the edge cases if it's not immediately easy to see them from the code.

There's even a simple macro to allow these tests to run alongside the clojure.test unit tests - which is exactly how I can see using it. There are some things that the unit testing system really works well for, and there are others that a shotgun approach would really help for as well. So a hybrid approach seems to be the best move.

Certainly interesting, and I'll put it into a project soon.

This morning I have nothing to do at The Shop, and so I decided to catch up on all the Faces work I needed to do in Apple's Photos app. It's getting to match the iOS Photos app a lot more, and while there are a few things to get used to, it's not bad, and if you just give it a few minutes, you're likely to find what you need right there in the app.

For example, setting the 'default' picture in a Faces collection used to be a scrolling and selecting deal - could be hard with a lot of faces, but now you look at all the pictures for a given face, and then right-click to set the one you want. Very simple. Very easy.

It's really a great little app.

I wanted to make a head-shot for my brother, and Faces had picked out a nice one, but it didn't allow me to set it as the contact. But it did allow me to duplicate the one picture... crop it... save it... and then the Contacts app allowed me to use that from the Faces for this contact. I mean it's just super simple.

At the same time, I have a somewhat unique problem in that two of my kids are trans. This means their names have changed - and not just a little. Once I got all the pictures under one name, then I simply clicked on the name in the Faces view, and changed it. Simple. And for me, that's a lot of pictures. So much easier than having to do each picture.

Finally, the camera metadata. Wow! I looked a a picture we all took a while back. It had all the metadata - not just the geolocation tags, but also the camera, the conditions... it's crazy what a camera will stuff into the metadata, and Photos accurately stores all that.

Very slick tool. Very. Thanks, Apple.

I had a meeting this morning with my manager and I was reminded of the verse:

For what will it profit a man if he gains the whole world and forfeits his soul?
-- Matthew 16:26

My manager is a consultant - been here two weeks - had a plan on Day 1 to change how The Shop was going to handle all data storage and reporting tasks. Based on what? Nothing. His experience. Not that he couldn't make it work, but the agreement to bring him in and trust him to implement the solution without knowing what the long-term costs would be is simply making a bad deal.

I understand the problems businesses can get into. I've been there. But I'm glad to say that I got out before I had to make any of those kinds of deals.

This guy is going to achieve what he wants, and give to management what they want. And when they get it, they will look at it, and say "This wasn't worth it." And they will be right.

But it will be too late.

I read on twitter about a bug in ZooKeeper found by the folks at PagerDuty. The story is quite remarkable, and reminds me that some companies still invest the time to get to the bottom of things - as opposed to just putting it off once a work-around is found. The level of detail and investigation they did is simply... inspiring. I'm stunned.

I'd like to work at a place where that kind of stuff is done. Not all the time, of course, as I'm sure they were all glad that it was over when it was over, but to be able to devote the time to solving the problem as opposed to stopping it - that's nice.

I don't imagine anything I'd do would hit this series of bugs. Too many components that simply aren't something I'd ever want to use. But it's nice to know someone is there digging deep.

This morning I don't have anything to do at The Shop - I'm waiting on people to decide what to do, and ordinarily that's a stressful time for me because some of the people blocking me aren't the most helpful of people I've run into. In fact, they just plain make me nervous. But this morning, I'm listening to Hootie and the Blowfish and I just know that this life of mine is in my hands - no one else's.

Not Liza's. Not the kids. No one - but me.

I can choose to feel beaten up, or not. I can choose to feel the victim, or not.

This isn't the first time I've felt this way, but it's been an exceptionally long time since I've felt this way. Just a silly smile on my face realizing that: No weapon formed against me shall prosper. It really is that simple - and it's really just that hard.

It's been so easy to feel beaten up - face it, there are people emotionally, and financially, beating up on me, so it's pretty easy to think that I am who they see me to be. They want to beat up on someone - they want a victim. So it's easy to see myself as they see me, and I then become the victim.

But the alternative is to realize that I have a code of ethics, a morality, that says I will honor my obligations, and then there's the divorce law, but that doesn't cover emotions. That doesn't cover honor. That isn't the sum total of who I am.

No, that's much more. So I can take the slings and arrows, because I choose how to handle them. They can be deadly - or they can be nothing more than the emotions ejected from others, and while I can see them, I don't have to internalize them.

Good start on the day!

There are very few things I'll walk two floors for - but a nice Diet Coke is certainly one of them. There is just nothing like a good Diet Coke for me. Coffee doesn't do it, Sprite, Lemonade - forget about it. So when I found this morning that the fountain on this floor was serving up soda water where once Diet Coke flowed freely... well... I had to grab some glasses and head downstairs. To the other fountains in the building.

Now I have a reasonable supply and I'm sure the kind gentlemen (or ladies) whose job it is to detect such outages will be right on the case and correcting this injustice as soon as possible.

But until then... Ah...

This morning I saw on HipChat messages from two folks at The Shop:

Chris M. said that you helped him to setup our new hardware. Guess what? I just ran a test ETL in new hardware, it is 5 times faster. The full MMS ETL cycle takes about 2-2.5 hours. In new server, it takes 0.5 hour. THANK YOU for whatever you helped Chris M. 🙂
-- Okji

and from Chris:

So, you were 100% right on the hardware specs for the Pentaho stuff.

Okji is running initial stuff now and it's insanely fast.

thank you for dealing with a stubborn asshat me through the ordeal and lighting a fire under my rear.

tbh I'd probably be flogging the dead virtualization horse at this point w/o that back and forth we had.

So yeah, thanks 🙂
-- Chris

I don't often have people sending me these kinds of notes for work I did for them. The problem was simple and obvious - to me, but if you have never seen the other way, you often think your way is the only way. I've seen it a million times. The point is to get them started, let them see, and then be very gracious when they thank you.

That last part is key.

You want to build up everyone - not just yourself. Help others feel good about what they did, and they will want to work with - or for - you again. It's simple. Who wants to be around someone that makes them feel bad about themselves? No one I know.

Interestingly enough, this is going to make things work a lot better for the short-term goals. There's a consultant at the shop, and this is going to make his Uber Plan much less attractive, and necessary.

One of the things I've always wanted in Postgres is the UPSERT - an INSERT that would update certain fields if the row (defined by the primary key) already existed. In the past, I've had to implement this as a custom function (stored procedure) in pl/sql, by checking for the existence of the row and then doing an UPDATE, or failing that, do an INSERT. It's workable, and it's not horrible, but it's also something that's in several other databases, and I wanted it in my database. 🙂

This morning I read a tweet that said:

and read the commit log message that described it. I love it! This is exactly what I've been hoping for.

The only question now is - When is it released?