What's it all about, Alfie?

Today was an interesting day. I was beginning to work on a new feature for this project this morning and then got pulled away from that to look at the seeming instability of one of the greeks for one of the portfolios. It seemed quite erratic, and clearly not driven by the market data or trading activity. It was really odd, to be honest, and it looked to me to be something in the calculations as opposed to the inputs.

So I started digging.

I had a heck of a time reproducing the input data simply because the SQL-styled input to a data source seemed to work when I had strings wrapped in single-quotes, but two columns of returned data were consistently zero. I dug and dug, trying to figure this out, but in the end, it was the fact that I tried double-quotes around the strings that solved it. With double-quotes around the strings, all the columns came back properly. Very odd, to say the least.

But then I had the data, and noticed that the normalization factors were very large - so large that they might really cause problems with the summing that was being done. Also, I checked on the positions and noticed that there were offsetting positions and that, along with the large normalization factors could quite possibly lead to loss of significance due to subtraction problems.

This took me several hours to come to.

But once I had a plan, I started re-working the calculations so that I don't get into these situations. I've done a lot of this in grad school, so it's not totally foreign to me, but the problems associated with what I had to work with made this difficult all the same. I needed to first calculate the net positions and then deal with a single position value per instrument. That's got to be a big winner. After that, I did the positive sums first, and then subtracted off the negative values. This should make these sums as well conditioned as they can possibly be.

I'll have to wait for tomorrow and see what the effects are on the stability of the numbers. The driving data is based out of London, and it's not ticking now, so I need to wait for the morning and then compare it to the production numbers. I should be a lot more stable, but if I'm not, then I'll keep going and get some stability. I have to, it's just too unstable and unreliable as it stands now.

Heck of a day.

[6/3] UPDATE: the stability was amazing. Really. Stunning. It was all I could hope for and then some. As the production app's numbers bounced around, the new development numbers were very stable. They moved, but they moved deliberately. That is a major improvement. Fantastic.

I had an interesting conversation with another developer today. He stopped by to tell me a problem he'd run into with the part of the code he's working on. He knew what he wanted to do, but he wasn't going to do it. He felt that it was better to give the users something as soon as possible, even though it was wrong, than asking them to hold off a while and build the right solution to the problem. It reminded me of the term I'd heard from another co-worker: Software Debt.

I found this article that does a reasonably good job of describing it. It's something that you have to be careful of every single day you write code. It's easy to hack something together and then say "I'll fix it later". The hard part is actually doing it.

This developer I was talking to was convinced that the solution to the problems created by these hacks added for immediate release was simply Eclipse. Yup, since Eclipse can refactor code, it's easy for him to go back and change everything about the code when he goes back into it later.

Now I'm not saying Eclipse isn't a great tool, but it's a refactoring tool, not a re-designing tool. And going back into the code presumes that you're back in the code. What happens if you don't need to go back in because the hack is only used once every six months, and at the time you notice that it's time to fix it, it's too late, and you're once again in the bind that something is expected by the users.

Rather than argue the point, I said "OK", and he's off to do the coding as he sees fit. I'm not his supervisor, he doesn't report to me, so it's a collaboration. In that vein, until it's my problem, it's his problem, and if that's the way he wants to handle his problems, then that's OK with me. In the end, we both know that it's wrong, and the right thing to do is to build something that's close to right (as close as possible), but still delivering close to the same timeframe as the hack. Get close, then come back and work it into the better solution.

But it's nieve in my book to think that it's just a matter of a little refactoring to change one system to another. Maybe it is that simple. More often than not, it's going to be some re-design as well as refactoring. It's not going to be something that's done unless there's a pressing concern to do it. While it's certainly possible that he's going to fix up this hack in the coming weeks, it's far more likely that much of it will stay and then some day we're going to be under so much software debt that we're going to be forced to throw it all away.

It's a shame. He agrees it's not the right solution, but is doing it anyway. Too bad.

While I've been dealing with a lot of jUnit testing issues this past week, for the last two days I've been dealing with the realities of trying to match the actual data from the system to the application I've got and that had turned out to be a lot more difficult than I had hoped.

The problem is classic: they have no comparison tools for the final stage in testing. They rely on eyes to look at one and the other over time to see if things "look right". This, as expected, can be very hit-n-miss, and I was getting a lot more of the latter than the former.

It's pretty easy to see if you're not even close, but if you're off by 10%, is that a mistake, or a sampling interval issue? Hard to say. If there's a lot of volatility in the sampled data, then maybe it's as close as you can expect to get. So I find myself wishing for fewer jUnit tests and better test frames for the end result. The fist are nice, but the second are critical.

In the end, the only way I'm going to get better test frame data is to be a part of it's creation. Unfortunately, I may have to do that.

Today has been a lot of messing around. I'm in the middle of adding a few new features to the app I'm inheriting from another developer and it's been a bit of a challenge. Over the last two days I've been adding the code and the unit tests to the codebase, and today it was time to get it all loaded up into the test server(s) and see how it runs. Things took a little longer than I'd have hoped, but all in all, it's understandable given that I haven't used these servers before today.

A few bumps in the road, but at least they had my favorite cereal in the kitchen this morning. It's amazing how something as simple as what you have for breakfast can really change your mood. There's got to be studies based on developers/creative types where they look at the effect of a new pad of paper, or a clean set of editor colors, or a new font - or cereal, has on your mood and how, in turn, that effects your productivity. In any case, it was a bit of a rough start, made smoother by those wonderful squares of goodness.

So I finally got things running and it came time time to compare the numbers between the code we added, and the application where we're getting the raw numbers. This application also does the aggregation of the raw numbers and that's what we're comparing. Did they match? Some. Never a good result.

I worked the rest of the day trying to get 29West working between a server of data and my app... didn't have a lot of luck, but did find out that speed/duplex problems are rampant even here, and my linux box was plugged into a switch that was improperly configured for auto/auto operation. That was fixed and helped a lot of things on my box, but not the 29West issues.

The data matching tests revealed that the formula I was using for the normalization of the values was wrong, and the right formula was a single factor different - not too bad, really. I fixed that up in the code and fixed the jUnit tests and things started to look better. But still not perfect.

Then the 29West guys noticed that it wasn't my box, but the sender's box that was having problems. So hopefully they'll get on that box and fix things up so he can send around the network. We'll have to see tomorrow.

Finally, my co-worker did the exhaustive testing of taking the raw data and using Excel manually aggregate it and compare it to the data in the app. There's only one problem with one instrument. It was just sprinkled throughout the portfolios so as to make it appear that there was more of a problem than there was. This was good news, but the day was over.

I'm sure glad I had those Mini Wheats.

I can really appreciate the case for unit tests. I have built them in many forms - 'test apps' and simple running tests that exercise the components, all the way up to jUnit tests for Java. There is a place for each, and I can see their value. But anything taken to an extreme can be bad. Very bad.

Take the case of unit tests I ran into today working with the original developer of an app adding a new feature to said app. It wasn't that hard... OK, it was harder than it should be because no one wanted to take responsibility for the data in the system.

OK... here's the problem.

Options expire. Futures expire. Options on Futures expire. But when an option on a future expires on the same date as the future, then the time it expires can be shifted. This is most easily solved by having the expiration date/time a table that understands this. But as with the Y2K issue, shortcuts are often taken in the initial stages of a project, and these things are thought to be "insignificant". Then, several years later (just like Y2K) the fixing of the problem is far far bigger than it would have been initially.

The system we were looking at had the future expiration date/time and the option expiration date/time. I said that the easiest way to know what to do is to look at the dates of these two expirations, if they are the same, then expire the option on the earliest of the future or option expiration. That way, if the future expired first, then the option is dead and has to expire at the same time. Simple.

If the future expiration date/time data were maintained properly in the system. Alas, it is not. Only the date component is maintained. So we had to put in a hack. I hate hacks. This was a double hack in my book because the author felt the best solution was to add a new constructor argument with a map of expiration times for 'double expirations' based on the future. This was then placed in a Spring XML file and then used in the code.

In truth, the change was only about 20 mins to do. We talked about alternatives for much longer than that (Given how I hate hacks). So the time required wasn't too bad.

But the jUint tests... oh... the tests.

We spent the next several hours updating jUnit tests to work with the new functionality. While I'm all for the testing ideas, when the testing code is bigger than the code under test, there's a hint you're doing something wrong. When you find a bug in the testing code, you know you're in trouble.

Testing code is meant to do Unit Testing. Not massive Q/A tests. Those are almost impossible to simulate and there's a reason that they exist. If you're spending two hours fixing up tests with 'fake' numbers, then chances are, you're making a mistake.

I still did it. I believe that consistency is very important - even if it's something I'd fight to the end of time to redo. If it's there, and if you're going to keep it, then by golly... make the new code work like the old.

But when unit tests go wrong, there's almost nothing worse. Almost.

I've been working all day on a single feature for this application at the Shop. It's got potential, but the way in which it's put together just screams 'Unnecessary Complexity', and I, for one, want to put an end to this kind of thing once and for all.

What is it about Java developers that makes them design such systems? What kind? you ask, I'll be glad to explain.

Java, as a langauge, is not evil. It's a tool. No more, no less. It's got a lot of nice features, and just as many limitations. It's not slow, per se, it's just not fast at everything. It's capable, descriptive, and works just fine - as long as you don't ask it to do something it's not meant to do. But that's not the real problem.

The problem, I think, is the way in which Java is evangelized by it's proponents.

Substituting design and planning for scores of small interfaces and "wiring it together" with something like Spring or even the equivalent of the Swing GUI tools, is not the right thing to do. IDEs like Eclipse and the rest allow these developers to put together projects that even they don't understand. This is a real problem.

Case in point. I was working with the author of this package today and asked him where a class was located. I'm a Vim/makefile guy for production software. It's universally available, easily transported, easily used on low-speed lines, and for all these reasons and many more, this is the most efficient toolset I've seen for the complete project lifecycle. So I asked him where the class was so I could load it up in Vim.

He didn't know.

He couldn't remember.

He had to have his Eclipse workarea opened up in order to find this class - that he was clearly very familiar with. After all, he mentioned it to me, and I just asked where it could be found.

This is but one danger - Package Explosion. Dozens of packages that have no reason to exist. He would be far far better served by simply thinking about the project and then laying out a few, well thought out packages and placing his code in these. Having this overly complex package layout is unnecessary, and while it's easy to use in Eclipse, it's a pain even to the developer that created it.

If I could pass these ideas on to this developer, I would. Sadly, he's likely too old to change his ways as he's a strong proponent of this type of development. However, in the hopes that someone someday might read this, here are a few pointers for coding that I've found exceptionally helpful over the years.

• If you can't remember where you put it, the structure is too complicated.
• If you require an IDE, then the project is poorly laid out and too complicated.
• Simplify, simplify... the best designs are the simplest.

I don't think this is exhaustive, but if you can stick to these few rules, you'll be far better off than what I've been dealing with all day. Holy Cow.

Well... they have helped me once again, those Googlers. They have released as stable Chrome 2 (actually 2.0.172.28), and with significant changes in the V8 JavaScript engine they are reporting a 30% increase in speed in JavaScript-heavy pages. Also, with the latest WebKit, page rendering is even faster.

I have to say this comes at a great time. I'm struggling with the size and memory footprint of my web app at work, and I have high hopes that this version of Chrome is going to be more stable, faster, and more memory efficient. Given that the problems are all in Google's hands (Chrome, Google Visualization API), I hope they have made real progress.

I've been working with 29West over the last few days and while I can see it's value, it's a little different than server-based messaging systems, and I can see why it's got advantages, and disadvantages. No need to critique it here... it's just what I have to use. But there's a consequence of using 29West's Java API on Tomcat and that's the fact that 29West's Java API uses JNI to get to the real C libraries under the covers.

With JNI, the shared libraries are loaded once in the Tomcat server, but if you want to change the code and remove and install the app again, you're in trouble because the shared library is not unloaded when the class loader is dropped. There's a lot of unhappy people about it, but in the end, there's nothing you can do. You have to remove the web app, shut down the Tomcat instance to drop the shared library, then start up Tomcat again, and then install the web app again.

I wanted it to be easier.

I got an interesting set of targets to do that. First, I need to have the start and stop targets, and they are simply exec targets to the locations of the startup and shutdown scripts:

  <target name="start" description="Start Tomcat application">
    <exec executable="${catalina.home}/bin/startup.sh"/>
  </target>

and:

  <target name="stop" description="Stop Tomcat application">
    <exec executable="${catalina.home}/bin/shutdown.sh">
      <arg value="-force"/>
    </exec>
  </target>

The value of the argument to shutdown.sh is that if I define:

  CATALINA_PID="/usr/local/tomcat/bin/.catalina.pid"

then catalina.sh will save the pid in the file and then on shutdown.sh it'll do a nice kill -9 on that pid and make sure it dies. This is really important because I need to kill the Tomcat instance and I need it to die right now.

Given that we have the standard remove and install targets from the default Tomcat Ant build.xml file, then all I need to do is glue these together:

  <target name="update" description="Update the application and bounce the server">
    <antcall target="remove"/>
    <antcall target="stop"/>
    <antcall target="start"/>
    <waitfor maxwait="3" maxwaitunit="minute" checkevery="500">
      <http url="http://localhost:8080/index.html"/>
    </waitfor>
    <antcall target="install"/>
  </target>

What's happening here is that we're removing the web app from the Tomcat instance, and then shutting him down forcefully. Then we're starting him back up, but since the startup.sh is asynchronous I need to wait until I can get a page back. When I can, then I'll install the web app again.

All in all, it's pretty sweet. It's not as nice a knowing that it's smart enough to unload the shared library, but there's nothing I can do about it. Actually... now that I think about it, I think it'd be better if the loader was smart enough to see that it's the same bloody file and link into it without an issue. But I'm clearly not as clever as these guys.

What I've got is workable, and given the limitations I have (29West and Tomcat), it's as good as I can expect to do for now.

Today I've spent a lot of time adding a new way of getting data into my web app. The Shop has standardized on 29West, and I've seen it in a lot of the smaller shops, but there's got to be something more to it... cost... performance... something because Tibco is the 800lb Gorilla, but not used in a lot of the shops around town. I'm guessing it's cost. 29West may be a lot more reasonable. In any case, it's different from the majority of messaging systems I've used in the past in that there's no central messaging server. You don't connect to a server and then get a topic. Instead, you simply connect into the system and then find your topic.

It's interesting.

It's using some multicast/broadcast system to talk to the other 29West clients on the network. You connect in a machine-to-machine scheme - publisher to subscriber. It's going to be more efficient with network switches as there's no single point that all messages have to go through. Makes sense. Looks a little like magic to simply "find" the publisher, but that's good coding for you.

If there's a downside, it's in the way you have to set up the communications. It's pretty nasty. You have to create a context attributes and then populate that with configuration values, and then from there you make a context, and then a context thread to handle the socket maintenance so you don't have to do it yourself, and then you find your topic for this context, and then either a source (publisher) or receiver (subscriber) for this topic and context.

It's all needed for maximum flexibility, but really... if you're going to make it look like magic, then why not just do the context, have it take a simple map of attributes - or nothing if the defaults will do, and have it fire up a thread. Then it's find the topic and you're off to the races.

You shouldn't need to make a source or receiver... those should be done in the context, or topic. Send and receive... can't get a lot simpler than that.

I was talking to one of the 29West guys here in the Shop, and it became much clearer to me as to why it was this way: 29West is C libraries and every one of their APIs are just shells onto these C libraries. In that case, it makes sense to have a not-very-object-oriented view of things. Reasonable, and you only need to code it up one.

I haven't had a lot of experience with the performance of it, but I'm guessing I will in the coming months. For now, I'm glad I have test sources and test receivers to make sure I can move messages around - box to box, window to linux, all that. Tomorrow I'm going to have to deal with a few nagging things about the impact on Tomcat, and see if I can't get data from the guy sending it to me to me.

I've been working on a few pesky HTML/CSS layout issues today and finally got them worked out. If you don't specify the widths of td elements in the page, then the browser is going to attempt a rough layout on the screen and then after it's all up there (placed), it goes back and does the fine-tuning on the location to move it to it's final location. Most times it's not a big deal, but when it's little movements, it's sort of annoying. I had a few of these today.

The lesson: specify everything. Don't make the browser figure anything out. Do it all beforehand and then you won't get this secondary movement.

What I had was a table with several things in it, and since the page was largely static once it was loaded (AJAX refreshed the data) it wasn't a huge deal, but it was annoying and I wanted to make this app as smooth as possible.

I started with defining the widths of the td elements. Not bad, but still not all there. Then I did a few more. And a few more. In the end, they were all specified, but still I was having movement on one page.

Then it hit me - maybe it was the table's cellspacing and cellpadding. So I used 1 for the padding and 0 for the spacing and then took into account the padding on all horizontal calculations. Bingo!

The HTML docs say the default is 0 for both these, but that's not what Firefox and Chrome were using. With these values and carefully counting all the pixels, I get exactly what I want.

It pays to be precise with layout.

Now it's perfectly set up initially and there's no secondary move. Just what I was hoping to have.