Chasing Down Numerical Method Issues
Today was an interesting day. I was beginning to work on a new feature for this project this morning and then got pulled away from that to look at the seeming instability of one of the greeks for one of the portfolios. It seemed quite erratic, and clearly not driven by the market data or trading activity. It was really odd, to be honest, and it looked to me to be something in the calculations as opposed to the inputs.
So I started digging.
I had a heck of a time reproducing the input data simply because the SQL-styled input to a data source seemed to work when I had strings wrapped in single-quotes, but two columns of returned data were consistently zero. I dug and dug, trying to figure this out, but in the end, it was the fact that I tried double-quotes around the strings that solved it. With double-quotes around the strings, all the columns came back properly. Very odd, to say the least.
But then I had the data, and noticed that the normalization factors were very large - so large that they might really cause problems with the summing that was being done. Also, I checked on the positions and noticed that there were offsetting positions and that, along with the large normalization factors could quite possibly lead to loss of significance due to subtraction problems.
This took me several hours to come to.
But once I had a plan, I started re-working the calculations so that I don't get into these situations. I've done a lot of this in grad school, so it's not totally foreign to me, but the problems associated with what I had to work with made this difficult all the same. I needed to first calculate the net positions and then deal with a single position value per instrument. That's got to be a big winner. After that, I did the positive sums first, and then subtracted off the negative values. This should make these sums as well conditioned as they can possibly be.
I'll have to wait for tomorrow and see what the effects are on the stability of the numbers. The driving data is based out of London, and it's not ticking now, so I need to wait for the morning and then compare it to the production numbers. I should be a lot more stable, but if I'm not, then I'll keep going and get some stability. I have to, it's just too unstable and unreliable as it stands now.
Heck of a day.
[6/3] UPDATE: the stability was amazing. Really. Stunning. It was all I could hope for and then some. As the production app's numbers bounced around, the new development numbers were very stable. They moved, but they moved deliberately. That is a major improvement. Fantastic.