Working on a Good, Fast, Cache for Messages
The last half of today I spent trying to figure out a really good way to incorporate a good, fast, caching scheme for these messages that are the heart of my ticker plant. The problem is that I know I'm going to be hammering the messages as fast as I can possibly take them, so a cache that locks is just right out. I wanted to make it simple as well - something parametric so it'd be easy to turn on or off and see what the difference in timing and throughput was.
The problem was, it wasn't really obvious how to do this with the given design I had.
Failed Attempts
I thought that it'd be best to put this into an existing class in the design - as low a level as possible. I was thinking maybe the component that handled simple message distribution to a list of registered listeners. It'd be nice there in that any component that "pushed" messages would be able to take advantage of this by simply "turning it on".
But there were a lot of conceptual problems. First, the messaging API used references for the messages, which is what I wanted to use, but that meant that in order to make the caching fit in without altering the API, I'd have to be making copies of the messages. That's not a good plan. Nothing good is going to come from copying a hundred thousand messages a second. That's a recipe for poor performance.
Next, if we changed the API to use pointers, and thought of the send() method as a "sink" of the message, then we'd have a nice, transparent, caching scheme, but the problem is that the messaging system relies on calling send() several times - once for each listener. This means I can't have the one method "eating" the instance as there'd be nothing left for the other listeners.
I messed around with this for a while, each time coming to the conclusion that the design I had wasn't really workable. I finally backed off and tried to think of the problem in vastly different terms.
What came to me was very simple, very clean, and in the end, far better than I'd ever have been able to do in the previous approaches.
The Component Approach
I began to think of the cache as something of a component as opposed to a capability of the objects in the design. I started to look at the cache as something that I could put as an ivar in the few components that needed it, and make it self-contained and easy to integrate and use, and all of a sudden things started to really look up.
Now I didn't have to worry about the passing of references and how many times something is called. In anything that deals with messages, it's possible to have a cache, and that cache will "eat" messages created on the heap. It will either delete them right away (if it's inactive), or it'll save the latest, and delete the older version, so as to keep the most recent in memory and yet at the same time handle all the housekeeping for the memory management of the created messages.
Most of the components that need this caching are things that create messages. Maybe it's coming in from a serialized data stream. In that case, the created message needs to be passed to all the registered listeners, and then deleted - or cached. It's similar for different "translation components", but the model fits very well. So I started coding it up.
I didn't get finished, but it's well on it's way, and it appears to have beaten all the problems that were really dogging me with the other approaches. I know there's more to deal with - like the "fast" part, but I'll deal with that in the morning.