Looking at Atomic Operations for Lockless State Management

cplusplus.jpg

I was listening to an interesting podcast today about lockless state management with atomic operations and got to thinking that there's a lot this can be used with if you spend a lot of time to get things into simple word-sized blocks. But there's a ton that can be done just as-is.

What I'm thinking about right now is the simple retain/release counters that I have on my instrument objects in the fast-tick server. Face it... they are about as simplistic as you can get. This article by IBM talks about the atomic operations in linux, and looking at a few include files, I can see that it's supported in the version we're using simply by including asm/atomic.h. Good news.

What I'll need to do is read up more on this and see if I can fit these easily into the retain/release code. If I can, then it's probably worth trying. The number of times I hit that code is non-trivial, and that's all kernel-space work. Better to be in user-space and not have the overhead. The code I have now looks a lot like this:

void Instrument::retain()
{
    // first, lock this guy up
    CKStackLocker      lockem(&mRetainReleaseMutex);
    // no up the count
    ++mRetainReleaseCount;
    // see if it's in the pool to be released, if so, remove it
    if (mRetainReleaseCount == 1) {
        CKStackLocker      lockem(&mReleasePoolMutex);
        mReleasePool.remove(this);
    }
}
 
void Instrument::release()
{
    // first, lock this guy up
    CKStackLocker      lockem(&mRetainReleaseMutex);
    // no decrease the count
    --mRetainReleaseCount;
    // see if it's an error
    if (mRetainReleaseCount < 0) {
        getLog() << l_error.setErrorId("Instrument.release")
                 << "the release count for " << mmSym() << " went negative! ("
                 << mRetainReleaseCount << ") This is a serious problem." << endl;
    }
    // see if it's at zero, and then if so, add it to the pool
    if (mRetainReleaseCount <= 0) {
        CKStackLocker      lockem(&mReleasePoolMutex);
        if (!mReleasePool.contains(this)) {
            // log that we're going to dump it in the trash
            getLog() << l_error.setErrorId("Instrument.release")
                     << "adding " << mmSym() << " with retain cnt="
                     << mRetainReleaseCount << " to garbage" << endl;
            // ...and then do it.
            mReleasePool.addBack(this);
        }
    }
}

The trick will be to do the entire method atomically. I can't just change the count. The checks are also critical, but maybe if I do the change, and keep the flag, that will work. Problem might be that another operation gets in the and wants to undo my work, but I come later... it's not an easy problem to solve with a single atomic operation on an int.

Certainly something to think about.