5 Cyclic frame buffers
5.1 Introduction
In some applications, in particular real-time processing of video images, it’s often desired to maintain a number of buffers, so that each buffer has a fixed size. In a video processing application, each such buffer contains one frame. This allows skipping frames or replaying them more than once, as necessary.
In a frame grabber application, an overflow condition can be handled by skipping one or more frames until there is a vacant buffer. For example, in a live view application, such overflow condition may occur when the viewing window is moved or resized. Dropping frames like this prevents the disruption of the continuous data flow from the video source, while maintaining a small latency.
In a frame replay application (e.g. a screen showing live output), the output image is repeated when there is no newer frame to display. This resolves situations where the source (e.g. a disk) momentarily stalls, causing the displayed image to freeze for a short while. While not completely graceful, it’s better than having the stream going out of sync. In many cases, the image repetition mechanism, although somewhat crude, works well for overcoming differences in frame rates, in particular when the output’s frame rate is considerably higher than the input’s frame rate (e.g. 30 fps to 60 fps).
This section discusses how the FIFO demo application, which was introduced in paragraph 4.4, can be modified to manage a set of buffers of this sort.
5.2 Adapting the FIFO example code
There are similarities between maintaining a cyclic set of frame buffers and a FIFO. In fact, if each byte in the FIFO represents a frame buffer, the readiness to read or write a certain byte in the FIFO is equivalent to the readiness to read or write an entire frame buffer.
For example, suppose a frame grabber application, where four frame buffers are allocated for containing the received image data. Suppose further that a FIFO of four bytes is set up to help managing these four frame buffers as follows:
The thread receiving the data starts from the first frame buffer, and continues to the next ones in a cyclic manner. Before starting to write to a new frame buffer, this thread checks that the four-byte FIFO isn’t full. After it has completed a frame buffer, it writes a byte into the FIFO and goes to the next one if the FIFO isn’t full.
The thread consuming the image data cycles through the frame buffer in the same order. Before attempting to read from a new frame buffer, it checks that the four-byte FIFO isn’t empty. When it has finished with a frame buffer and is ready to go to the next, it reads a byte from the FIFO.
By sticking to this convention, it’s guaranteed that the thread receiving the data will never overrun a frame buffer that hasn’t been consumed, and that the consuming thread will never attempt to read from a frame buffer that contains invalid data. As a matter of fact, the number of bytes in the FIFO represents the number of valid frame buffers in the set.
Note that the values of the bytes written and read make no difference, so there’s no actual need to allocate these four bytes of memory and store data in them. Only the FIFO’s handshake mechanism plays a role.
Hence, the FIFO API outlined in paragraph 4.6 can be adopted as is:
-
Call the function fifo_init() with the size parameter as the number of frame buffers (recall that size can be any integer). fifo_init() will allocate and lock memory for the FIFO, which will never be used (since each bytes just symbolizes a frame buffer). This waste of memory is negligible, but the relevant portions in the code can be removed to avoid future confusion.
-
Call the function fifo_request_drain() to get a frame buffer to read from. info->position will contain the index to the frame buffer to use (numbering starts at 0). If no frame buffer is ready, fifo_request_drain() will sleep until there is.
-
After reading from the buffer, call the function fifo_drained() with bytes_req=1.
-
The functions fifo_request_write() and fifo_wrote() are called in the same way by the thread writing to the frame buffers.
-
FIFO_BACKOFF should be set to zero. There is no point for this feature with frame buffers.
5.3 Dropping and repeating frames
Let’s take the case of a continuous source of image frames which must never reach the state of overflow, given that the data comsumer may not always collect the data fast enough.
The idea is to prevent blocking on the thread, which transports data from the data source to the frame buffers. To achieve this, the following sequence should be looped on for each incoming frame:
-
Call the function fifo_request_write() to find out which frame buffer to write to
-
Write to the frame buffer pointed at by info->position
-
When done writing, call the function fifo_request_write() again. This function call will surely not sleep (block), because no buffer has been reported as written to, since the previous call.
-
If fifo_request_write() just returned a value larger than 1, call the function fifo_wrote() (with req_bytes=1, of course). A subsequent function call to fifo_request_write() will surely not sleep (block), because there were more than one buffer to spare, and only one was consumed. In fact, the next function call to fifo_request_write() can be substituted by just picking the next frame buffer.
-
On the other hand, if fifo_request_write() returns just 1, don’t call the function fifo_wrote(). Instead, use the current buffer again on the next loop executing for accepting incoming data, or just drain a whole frame from the data source to no particular destination.
Since this usage prevents blocking, it’s possible to delete the while() loop in the implementation of fifo_request_write(), as it is never invoked. Further code reduction is possible by removing the relevant semaphore, as well as its initialization and destruction code. Leaving them in the code has a minimal effect, so this optimization is a mostly a matter of keeping the code readable.
A similar approach can be taken to repeat frames on the thread writing from the FIFO: Call the function fifo_request_drain() again just before calling the function fifo_drained(), and repeat the current frame if it returns less than 2.