What's it all about, Alfie?

One of the nice things about using JRuby is being able to use Warbler to package up all the files and scripts into a single jar for easy deployment. One of the problems with that is that some of the common coding statements don't work the way you expect them to - but they do work.

When I had an issue with reading a CSV file in a jar, my solution was to "go to the metal" and work it out in Java. This worked, and it was OK, but it wasn't something that was transportable to a non-JRuby environment, and I wanted to have that. So this morning I tackled just that.

The thing that's been bugging me was that Configulations works just fine in our code - jar or no-jar, so why was that working and the file reads of CSV files not? What I needed to do was to look seriously at how they were being used.

We were using Configulations in the following snippet:

  require 'configurations'
  require 'date'
 
  AppConfig = Configulations.new(File.dirname(__FILE__)+'/../config')

I know that JRuby 1.7.0 doesn't like the use of the '/../' in the path in a jar as it tries to "walk" the structure, so I changed this to a slightly less round-about way:

  require 'configurations'
  require 'date'
 
  AppConfig = Configulations.new(File.dirname(File.dirname(__FILE__))+'/config')

This allows JRuby to handle the parsing of the path as it sees fit and I don't have to worry about moving around within the jar file like I can in a filesystem.

And then it hit me… how we're using the filenames in the reading of the CSV files:

  require 'csv'
  require 'json'
 
  module FileUtility
    DATA_FILE = 'lib/sales/histData.csv'
 
    # ...
    if File.exists?(filename)
      File.open(filename) do |file|
        contents = file.read
      end
    else
      # ...
    end
 
    # ...
  end

and it stood out as clear as can be - the path was wrong. We were looking for these CSV files in a directory relative to the existing directory. How was JRuby to know that we wanted to look within the jar? Impossible.

The Configulations example worked because it used the dirname() method relative to the existing file - which is in the jar. That means that we were the ones telling JRuby to look in the jar (or on the filesystem), and it was all about the path we prepended to the beginning of the file we wanted to load.

There was no need to have the Java solution - we just needed to be more careful with the location of the CSV files. What we have now is far simpler:

  def self.read_file(filename)
    contents = ''
    File.open(project_root + '/' + filename) do |file|
      contents = file.read
    end
    contents
  end
 
  def self.read_csv(filename, key)
    res = {}
    CSV.parse(read_file(filename), :headers => true).each do |rec|
      k = key.map { |c| rec[c] }
      res[k] = rec
    end
    res
  end
 
  def self.read_json(filename)
    JSON.parse(read_file(filename))
  end
 
  def self.project_root
    @root ||= File.dirname(File.dirname(__FILE__))
  end

where once again, we use the "double dir" method chain to get the parent's directory, and use that to know that it's the root of the project - based on the location of this file in the project. This is far, far simpler than we had in the past, and it's removed all the silly Java code that wasn't really necessary in the first place.

It's important to realize you solved a problem, but not in the way it was intended. I'm glad I went back and fixed this. Very glad.

One of the things that's really quite limiting about the current demand forecast data from another system at The Shop is that it's not a month-by-month demand projection. They took the year's sales data and then extrapolated for the next month. But in doing so, they have removed any monthly/seasonal trends so that the demand for Boat Tours in Chicago is the same in July as January. Clearly, absurd. Unfortunately, there doesn't seem to be the will to push a change in and fix the Demand in a reasonable timeframe. So I had to add in some kind of demand adjustment based on the month and some sense of the time to close a deal.

What I came up with was a series of factors - one per month, that could be individually adjusted in a GUI - like an equalizer, and then we'd take each of these and multiply the given demand by this factor and divide by 100 (converting it into a percentage). In this way, we can ramp things up, and shut them down, a month at a time.

If they want to boost something up, they can pretend it's seasonal, and use move everything up to 200%… if they want to suppress something, then move all months to 25% - simple. It's a great plan, and we can "seed" the data with the sales data by service, by month.

What I need to do is to start with the code and then work back as I'm not about to do the GUI first - I'm no artist, and that's what this is going to take to make it look really nice.

This morning we once again had a series of production problems caused by Salesforce.com - this time, it was that a class had been deployed to the production instance and the permissions hadn't been set properly. How I figured this out - I don't know. I mean, I know how I figured it out - I looked, but to look was a stroke of luck.

I certainly believe in things we can't see and touch, so I'm not against divine intervention, but I just have no idea why He'd want me to figure this out so quickly. In any case, all the calls to updating the merchant data were failing, and the error was not the same one as I'd seen previously, so seeing no one in the Salesforce support group in at 7:00 am, I decided to start digging and see if I could find the problem.

Thankfully, I remember the support crew checking these in the last go-round, so I thought I'd give it a go. Turns out I was right. Still took me until 9:00 am to get everything re-run for the day. Nothing I could do about it, but without checking on their part, and them coming in later, there's not much else I could do.

I just saw a tweet and immediately started to giggle:

I hit the link and started reading. The changes are OK, but nothing substantial like keeping things open and allowing image sharing. Then, at the bottom of the article, they said:

We’ve made a bunch of other improvements to the app and many more are on the way. Thanks everyone for your great, constructive comments. We’re listening. We’ve also made the old version of Skitch available for those that want it. Stay tuned, our app to help you communicate and share your ideas visually is just getting started.

HA!

Evernote looked at the horrible ratings on the App Store, and they relented to leave the older version up. I immediately downloaded it on my MacBook Pro and it fixed all the configuration issues I've been trying to get around. This is excellent news!

Now, all I need is to set up a decent WebDAV server, or just leave it as FTP and I'm good to go. Also, I have CloudApp and that works as well.

Today my day started out with a rather early meeting for my work here at The Shop - 9:30 am, and it was about an interesting topic - the seasonal adjustment of the demand data we're using in our project. The problem is that the demand data is based on the previous 12-months historical sales data - all annualized. This means that boat tours should sell as well in July as December. Nope. So how to do this?

I'd already suggested in an email that we make a simple web app that allows the right people to add the important location and taxonomy data as well as a simple 12-month segmented graph (or even sliders or dials) so that the users could control the seasonality for their division and their sales reps. It makes sense - boats trips in Arizona are not down in December, so we need to really look at all the factors that might contribute to the seasonality of the demand.

Then we need to overlay this on the incoming demand and we're in business.

There's also the need for a manual demand entry screen where the users can input demand that they anticipate, and let them run with that. All sounds pretty decent. Very do-able.

But the proposal was that we base it on some Google Doc and parse out the data from a spreadsheet.

That's a horrible idea. I've had to do it so many times, and it always becomes a nightmare. The project manager thought it would be "easy" and "fast"… but all it really does is move the development effort from building the UI and data integrity checks to the data parsing and processing. There's no savings here. But Holy Cow! That was a 30 min meeting that took 90 min because this guy could not get the idea out of his head that this wasn't the way to do it.

I consider myself a decent communicator, but I've come to a loss with this guy. I think he's just not a good listener, but who knows. In any case, at the end of the 90 min meeting I was able to get some support from another developer manager and I think we're going to do the "Right Thing". But it was not easy.

Now I think I'm going to have to come up to speed on Rails, as I think we're going to build this in Rails, but at least it's going to be done Right.

Looks like we have a new version of Google Chrome dev this morning - 23.0.1271.10 with a sparse, but informative list of release notes. With this update, it appears they are about to jump to 24.x - but we'll have to wait and see. They may all be working on getting iOS Maps out 🙂

In any case, fixes for Flash are always good (I really dislike the implementation of the Flash interpreters I've seen from Adobe), and while I'm not a Windows 8 fan, it's nice to throw them a bone once and a while.

Today has been one of those days… it started out with some wonderful steel cut oatmeal at the cafeteria… I had no idea they had that! It was wonderful. But from there on, it was a steep descent downhill.

We are coming off the horrible morning and then trying to get a good number of features and fixes into the code for the afternoon release to production. I like to keep releasing something off master to production each day as that allows the users and project manager to see some visible progress each day. Big or small, it makes little difference to be able to point out the changes we have made based on their feedback and requests.

Often times, though, this means doing a lot of nasty work to follow up with people that dropped the ball (but doing so in a way to make them feel like they are doing you a favor), cleaning up problems and messes left by others, and all the little ick work that comes with software development.

Today has been that day for me.

I'm glad it's over.

On the up-side… we are releasing something far better than yesterday, and the really great ones never loose sight of the fact that it's all in the details.

This morning I've been fighting off a few things and while I'm doing better at being able to handle the slings and arrows these days, I was given a wonderful reminder from twitter:

In 100 years I won't be here. My kids won't either. Their kids? Probably, but that's only if they live well. There may not be a soul on this planet that remembers what I'm doing in this life, and that's OK.

Who was working in the machine shop (the high-tech equivalent of today) in GE's plant 100 years ago? No idea. He worked hard, tried to raise a family, be a good husband, and some did well, others not so much. But today, they are long forgotten.

Carpe Diem. 'Nuff said.

This morning I saw this on Daring Fireball, and it reminded me of the recent conversation I've had with a guy that contracted me to do some work for a Bank and said that it'd be worth $25k to him. So I did it. It took me about 2 weeks, but it was a lot of fun getting back into OS X coding that I didn't mind the time.

I actually made it more flexible than he/they needed. I could sync up and down data between multiple devices and it always makes sure to send minimal packets of information. It's not Rocket Science, but it's nice, and I'm really proud of it. I put it up on a private repo on GitHub and let him see it.

He wanted to use it for something else, and when he asked how much that was worth, I said: "$25k".

"What? You said it'd only take a weekend to make the customizations I asked for."

"Yes, but that's not the point… it's the value the code represents."

I just wish I'd had this quote from Picasso to reference him. I have it now.

"Five thousand dollars," the artist replied.

"B-b-but, what?" the woman sputtered. "How could you want so much money for this picture? It only took you a second to draw it!"

To which Picasso responded, "Madame, it took me my entire life."

I love it when I'm able to work with people that see the same Big Picture as I do. It doesn't happen often, but when it does, it's almost magical. The next best thing is to work with someone that can see some Big Picture - even if it's different from mine, and then we can hash out the differences and come to some accord with how to get to that endpoint.

Some of the most frustrating people to work with are those that simply are incapable of seeing the Big Picture. Maybe they don't think in those abstract terms. Maybe they don't think there is such a thing. Maybe they aren't looking at what's being done as much as how it's done. For whatever reason, I'm in the midst of trying to make someone see the Big Picture, and it's something that before too long, I'm just going to give up on.

This isn't a critique of the person… it's the old adage:

Don't try to teach a pig to sing - it only frustrates you and annoys the pig.

if a person isn't going to see what you want them to see - for whatever reason, then it's time to just stop and let them be the person they want to be. If they have the capability of seeing it, and just don't want to, then maybe, someday, they'll change their mind and come to you seeking out your advice then.

If they can't see it at all, you're annoying them, and if at some point in the future, they want to see try again, they will again seek you out, and try again.

But until then, it's just a problem - for you and for them. Better to accept them as they are and move on. No amount of cajoling, pleading, arguing is going to make an adult change their mind. They have to come to that decision themselves.

So I'm trying to convince myself that the right time to let go is now.

Right now.