Tricking a Tricky Threading Problem
This afternoon I've been tracking down a good solution to a nasty threading problem. This part of my ticker plants is the UDP receiver and it tries to get the UDP datagrams off the socket and into a buffer as fast as possible. To that end, I've got a single-consumer, single-producer, lockless FIFO queue that should be thread-safe as the 'head' and 'tail' are volatile and there's only one thread messing with one of these guys at a time.
But that's just the theory. Here's what the code looks like:
template<typename Element, uint32_t Size> bool CircularFIFO<Element, Size>::push( Element & item ) { uint32_t nextTail = increment(tail); if (nextTail != head) { array[tail] = item; tail = nextTail; return true; } // queue was full return false; } template<typename Element, uint32_t Size> bool CircularFIFO<Element, Size>::pop( Element & item ) { if (head == tail) { // empty queue return false; } item = array[head]; head = increment(head); return true; }
Here's what happens: I'll be running just fine and then the call to pop() will return true and because of that, the value (a pointer) will return as something. This presents a real problem. If it returns a NULL, that's easy to deal with. Problem happens when it returns junk.
Ideally, it wouldn't return a NULL or junk, but coding for that has turned out to be harder than I thought. First, I can just check for a NULL or what I think of as "junk" data, and not delete that pointer, but what happens when it returns "junk" that's not fitting my pattern of "junk"? Well... I'll delete it and BAM! SegFault.
Not easy.
I believe the problem is one of compiler preference. The data in the class is defined as:
volatile uint32_t head; volatile uint32_t tail; Element array[Capacity];
where the lack of the volatile keyword is the big deal here. What I need to do is to make the data look like:
volatile uint32_t head; volatile uint32_t tail; volatile Element array[Capacity];
and then correct all the castings in the code to make them work properly.
I've got something to compile that looks like:
template<typename Element, uint32_t Size> bool CircularFIFO<Element, Size>::push( Element & item ) { uint32_t nextTail = increment(tail); if (nextTail != head) { array[tail] = *((volatile Element *)(void *)&item); tail = nextTail; return true; } // queue was full return false; } template<typename Element, uint32_t Size> bool CircularFIFO<Element, Size>::pop( Element & item ) { if (head == tail) { // empty queue return false; } item = *const_cast<Element *>(&(array[head])); head = increment(head); return true; }
We'll have to see how this runs.