The Seemingly Ever-Changing Views of Management
In the recent weeks one of our data vendors has contacted us about our use of their API. Specifically, they created this API for themselves, as their product runs on it, but they wanted to make it possible for users of their product to also get at this data in a programmatic manner. It was a nice thought. We looked at this and decided that it would be even nicer if the general users of this data didn't have to individually write to the API. Maybe even add a cache to the data received so that for each day, a single piece of data would only be fetched from the provider once - then after that, subsequent requests would hit the local cache. It's an excellent idea, a wonderful product, and now they are interested in charging us more than $1,000,000/year because of the data we get from this.
The reason for this is that this vendor decided to un-bundle the API from the product they sell that uses this API. They decided this because, of course, people were using it and they saw it as an additional revenue stream for their company. I can't blame them, it's capitalism as it's best - make something someone wants and then charge for it... if they pay, then you have a winner, if not, then you are out the work to make the product. So they came to us and gave us a pricing model for the data we're getting and it averages out to a little over a million dollars a year.
Given that the bundled product we got originally is about $2,700/month - this represents a large increase, and it's likely that the business will simply not see that this data is worth seven-figures a year. Again, that's their job - cost/benefit analysis... it's a necessary part of all businesses.
The problem comes in that it's not this easy - the vendor isn't just offering us an upgrade path, they are also (at the same time), holding a gun to our heads. If any of the trading business requires this data to function (as some do), then turning it off is simply not an option. Migration is the key, but migration to what? We'll have to look into the different competitors of data and look to see what each has and the cost, etc. Then work that into the systems one by one and then we'll be ready to turn off the high-priced original vendor. It's more than an afternoon of work.
The other side of the problem is that the management here has known about this issue for years. Yes, years. Each time they talk down the vendor from the ledge, and they agree to let us continue to use the data under the existing contract. When we start to get too much data from them, we find alternatives and migrate a few applications from that one source to another under controlled conditions and hitting the easiest, biggest offenders first. Makes a lot of sense. Usage of the data drops, they are happy, and life goes on.
But not now. No, they are convinced that the vendor is out to get the full price and lock down all the data. The problem with the lock-down stance is that there's no way to really do it unless we shut down all usage of the data. Imagine, I use the data in a spreadsheet. I save that spreadsheet on a network drive that others look at. That's redistribution of the data and they won't allow that. The only people that can see the data are the people they license to see the data - no exceptions. And if there's a possibility that someone else might see the data, then that has to be shut down. Their latest position is very restrictive, and to follow it to the letter will mean you really can't use the data for anything "saved" in any way, shape or form.
Clearly, we need a new source for data, but management is taking the attitude that we need to "spread out" the usage and then pay for what we really use - sticking to the letter of the license. But there won't be any data that we can use to the letter of the license as it's a server, and therefore might serve up data to something that possibly makes it available to someone not intended to see it.
It's dumb, but I'm not going to fight it. If they want to shut it down, fine. They aren't going to - not without a replacement, and when that replacement arrives we'll simply slot it into the server alongside the old, restrictive one. It's going to be a mess, but they seem to have given up on talking to the vendor on this. Maybe it's not possible, but I can't imagine a vendor that would take nothing over something - which is what they'll get if we shut this down. But then again, they could be trying to shut it down on their end, and this is their way of doing it. Vendors have been known to do dumber things.