Streaming buffers, windows, and framing

"A sensor speaks in an unbroken whisper. My models only listen in complete sentences. The buffer is where I decide when a whisper becomes a sentence."

A Patient AI Agent

This section builds directly on the ideal sampling grid and the Nyquist limit from Section 3.1, and it assumes you have internalized the timing defects (jitter, gaps, loss) from Section 3.4, because a buffer is exactly where those defects must be reconciled before anything downstream sees a frame. The spectral consequences of the shape of a window (Hann, Hamming, and their leakage tradeoffs) belong to Chapter 7; here we care about the timing and mechanics of windowing, not its frequency response.

The ring buffer: absorbing the producer-consumer mismatch

Get this one structure wrong and the failure is invisible: a fitness wearable drops the exact samples that held a fall, or a voice assistant clips the first syllable of your command, and nothing downstream ever flags that anything went missing. The small piece of plumbing that stands between an unrelenting sensor and a model that pauses to think is what makes those losses avoidable. A sensor is a producer: it writes samples at its own cadence, often from an interrupt or a direct memory access (DMA) transfer, on its own thread. Your model is a consumer: it wakes up periodically, grabs a chunk, and runs. These two rates almost never match instant to instant. The producer may deliver samples in bursts of 32 while the consumer wants blocks of 256; the consumer may stall for 40 ms during a garbage-collection pause while the producer keeps writing. A ring buffer (circular buffer) is the standard structure that decouples them. It is a fixed-size array with a write pointer and a read pointer that both advance modulo the array length. The write pointer overwrites old data only after the reader has consumed it, and the buffer allocates no memory in the steady state. The fixed size is the point: on an embedded target you cannot let a queue grow without bound, and the ring buffer gives you a hard, predictable memory ceiling.

The ring buffer's capacity is a design decision with real consequences. Too small and a brief consumer stall causes the write pointer to lap the read pointer, silently overwriting unread samples: this is buffer overrun, and it manifests downstream exactly like the packet loss of Section 3.4, except that you caused it yourself. Too large and you pay for it in latency: a sample sitting near the back of a deep buffer waits a long time before the consumer reaches it. Size the buffer to cover the worst-case consumer stall you are willing to tolerate, and no larger. A useful rule of thumb: the buffer must hold at least the number of samples produced during your longest expected scheduling gap, plus one full frame. In short: a buffer buys safety with time, so hold exactly enough samples to outlast your worst stall and not one more.

Windows and framing: length, hop, and overlap

Once samples are safely in the buffer, framing extracts fixed-length windows from them. Three parameters define the policy. The window length \(L\) is how many samples each frame contains; it sets the temporal context the model sees and, through \(L/f_s\) (the window's length in seconds, where \(f_s\) is the sampling rate carried over from Section 3.1; a longer window in seconds resolves finer frequency detail), the frequency resolution available to any spectral stage. The hop \(H\) (also called stride or step) is how far the window advances between consecutive frames. The overlap is the shared region, \(L - H\) samples, usually quoted as a fraction \(1 - H/L\). When \(H = L\) the frames tile the stream with no overlap; when \(H < L\) they overlap; \(H > L\) would skip samples and is almost always a bug. These three numbers do more than describe a single frame; together they define a segmentation policy, and the simplest such policy is worth naming outright. Figure 3.6.1 makes the three parameters concrete: successive windows of length \(L\) slide across the incoming sample stream, each advanced by a hop \(H\), and the \(L - H\) samples two neighboring windows share are the overlap.

Fixed-length framing is the policy of slicing a stream into equal-sized, regularly spaced blocks defined entirely by \(L\) and \(H\), independent of what the signal is doing. It matters because it gives every downstream stage a predictable, uniform tensor shape and a constant frame rate, which is what makes batching, feature extraction, and real-time scheduling tractable. Mechanically, a frame is emitted the instant \(L\) contiguous samples exist and the read pointer then advances by exactly \(H\), so timing is governed by sample arithmetic rather than signal content. Use fixed framing when you need steady throughput and stationary-within-a-frame statistics (signal properties that stay roughly constant across the span of one frame); prefer event-triggered or onset-based segmentation instead when the phenomena of interest are sparse and bursty (a heartbeat, an impact, a keystroke), because a fixed grid then wastes compute on silence and still splits events across boundaries.

Overlap exists to solve a real problem: an event that straddles a frame boundary is split across two windows and may be under-represented in both. Overlapping frames guarantee that any event shorter than the overlap appears whole in at least one window. The cost is compute. The number of frames per second is \(f_s / H\), so halving the hop doubles how often your model runs, and buys zero new information while doing it: the overlapped samples are ones you already had. This is the master tradeoff of streaming framing (the third term below, the latency floor \(L/f_s\), is unpacked in the paragraph just after the equation):

That last inequality is the one people forget. A causal frame cannot be emitted until its last sample has arrived, so a window of length \(L\) imposes an inherent latency of at least \(L/f_s\) regardless of how fast your model runs. Doubling \(L\) to get finer frequency resolution also doubles your minimum response time. Human-activity recognizers commonly use windows of 1 to 2 seconds with 50 percent overlap, a choice examined closely in Chapter 26; that already commits the system to at least a one-second reaction time before any clever engineering.

A minimal streaming framer

The code below implements the core pattern: a bounded buffer that accepts arbitrary-sized chunks from a producer and emits fixed-length, overlapping frames whenever enough samples have accumulated. It carries a running sample index so every frame knows its own start time on the global grid, which is what lets a later stage align frames across sensors as in Section 3.3.

from collections import deque

class StreamFramer:
    """Turn variable-size input chunks into fixed-length overlapping frames."""
    def __init__(self, length, hop, maxlen):
        self.length, self.hop = length, hop
        self.buf = deque(maxlen=maxlen)   # bounded: old samples drop if we stall
        self.start = 0                    # global index of buf[0]

    def push(self, chunk):
        """Feed a chunk of samples; yield every complete frame it makes ready."""
        n_before = len(self.buf)
        self.buf.extend(chunk)
        # If maxlen clipped us, advance the start index by however many were dropped.
        dropped = max(0, n_before + len(chunk) - self.buf.maxlen)
        self.start += dropped
        while len(self.buf) >= self.length:
            frame = [self.buf[i] for i in range(self.length)]
            yield self.start, frame          # (start_index, samples)
            for _ in range(self.hop):        # advance by one hop
                self.buf.popleft()
            self.start += self.hop

framer = StreamFramer(length=256, hop=128, maxlen=1024)   # 50% overlap
for chunk in [range(0, 100), range(100, 400), range(400, 610)]:
    for start, frame in framer.push(chunk):
        print(f"frame @ {start:4d}  len={len(frame)}")

Running it turns three ragged chunks (100, 300, and 210 samples) into evenly spaced 256-sample frames at indices 0, 128, 256, and on. The buffer hides the producer's chunking entirely: the consumer sees one clean, regular frame sequence, however lumpy the arrivals.

Framing pitfalls that leak or lie

Two mistakes recur often enough to name. The first is overlap-induced leakage across a train/test split. Suppose you frame first with 50 percent overlap and split into train and test sets afterward. Adjacent frames then share half their samples, so a test frame can look nearly identical to a training frame. Memorization, not perception, then inflates your reported accuracy. The cure is to split by contiguous time segments before framing, a discipline developed fully in Chapter 5. The second is edge handling: the first \(L-H\) samples of a stream cannot form a full frame, and neither can the final partial window. Zero-padding those edges injects a discontinuity that a spectral stage reads as spurious high-frequency energy. Decide explicitly whether to drop partial frames, pad them, or carry them forward, and record which; never mistake a silently padded frame for a real observation. This same care distinguishes a robust streaming pipeline from a lab demo, and it connects to the streaming-inference machinery of Chapter 60, where frames feed a model that must never stall the sensor.

What's Next

Section 3.7 steps back from moving individual samples to governing the whole acquisition: the protocols that carry sensor data reliably, and the metadata (sample rate, units, timestamps, calibration, and provenance) that must travel alongside every frame so that a recording made today stays interpretable, and trustworthy, years from now.