Another Tough Day of Slugging Through Testing
I've spent most of this morning slugging through tests to see how the performance of the process is effected by different things it has to do. For example, writing results to Salesforce… and writing processed data to CouchDB… It's not good news, that's for sure, but it's data, and you have to respect what it's trying to tell you - even if you don't like what it's saying.
There's a story here, and figuring it out is the real challenge.
I've got a few ideas on what to test next, but the overall scheme isn't looking good. We use a lot of data, and reading and writing it is just a huge part of the process. The reasons for writing it back to Salesforce are obvious, but a case could be made that this is the wrong direction to take, and we should remove ourselves from a dependency on Salesforce and just make everything stand-alone.
There's a lot I like about this idea - it still has only one copy of the data, it's just no longer in a third-party application that we can't really access efficiently. It does mean that all the supporting tools (report builders, query tools, etc.) that are in Salesforce would have to be duplicated outside in the system we built, but that's a one-time cost as opposed to the continual cost of moving the bits from there to here and back again.
I know that in Finance this would not have been a long-term solution. But we're not in Finance anymore, and I'm trying to go with the flow here and see what plays outside of Wall Street. So we'll see… maybe it stays, maybe it doesn't.