Irregular and asynchronous sampling

"I was trained on a metronome and deployed at a jazz club. Nobody told the samples to arrive on the beat."

A Rhythmically Challenged AI Agent

This section assumes you understand sampling, jitter, and clock synchronization from Chapter 3, and that you can build the recurrent and convolutional encoders from the earlier sections of this chapter. It also leans on the leakage-safe splitting discipline of Chapter 5, because irregular data offers new ways to leak the future into the past.

Where irregularity comes from, and why you cannot ignore it

Irregular sampling is not one problem; it is a family. Event-driven sensors emit a reading only when something crosses a threshold, the extreme case being the event cameras of Chapter 46. Power-managed sensors duty-cycle, sleeping for a variable interval so the gap itself encodes a scheduling decision. Lossy transport, typical of Bluetooth Low Energy wearables, drops packets under radio contention, leaving ragged holes in an otherwise uniform stream. Asynchronous multi-rate fusion combines a 100 Hz IMU, a 1 Hz GPS fix, and an event-triggered barometer, each on its own clock, with no single grid onto which all channels naturally fall. And human-triggered records, most clinical data, appear only when a measurement was clinically warranted, so the sampling pattern is itself a signal about patient state.

The tempting fix, resample everything to a uniform grid and pretend, is dangerous for two reasons. First, upsampling by interpolation fabricates measurements: a linear fill between two blood-pressure readings twelve hours apart invents a smooth trend the patient may never have had, and a forward-fill can carry a stale value across a state change. Second, the interpolation kernel can straddle the split boundary and leak future information backward, exactly the trap flagged in the forecasting discussion of Section 14.4. The gap length is often the most predictive feature you own, and resampling deletes it.

Three ways to make a model time-aware

The cheapest intervention is time as a feature: append the elapsed interval \(\Delta t_i = s_i - s_{i-1}\) (where \(s_i\) is the timestamp of sample \(i\)) as an extra input channel, and often the observation mask alongside it. A vanilla GRU or TCN can then learn to modulate its update on the gap. This is a one-line change and a strong baseline, though the model must discover the right functional form of the gap on its own.

The principled recurrent answer is a time-decayed state. The GRU-D and T-LSTM families make the hidden state forget as a function of elapsed time: between two observations the memory decays toward a learned baseline, fast for channels that go stale quickly (heart rate) and slow for those that persist (a chronic diagnosis). A common decay is

with the decayed state \(h_i^{-}\) fed into the usual gated update. When \(\Delta t_i\) is small, \(\gamma_i \approx 1\) and nothing changes; when the gap is long, the state relaxes toward its prior, which is exactly the behavior a Kalman prediction step exhibits between measurements in Chapter 9.

The most expressive answer models the hidden state as a continuous-time trajectory. ODE-RNNs and latent-ODE models evolve \(h(t)\) by a learned ordinary differential equation \(\mathrm{d}h/\mathrm{d}t = f_\theta(h)\) between observations, then jump the state at each arrival with a standard recurrent update. Because the dynamics are defined for every real \(t\), the model is genuinely grid-free: you can query it at any timestamp, interpolate and extrapolate within one framework, and handle wildly uneven spacing without a single imputed value. This continuous-time viewpoint connects directly to the linear-time state-space models of Chapter 16, several of which are discretizations of exactly this kind of continuous system.

import torch, torch.nn as nn

class GRUDecayCell(nn.Module):
    """Minimal GRU-D-style cell: hidden state decays with the elapsed gap."""
    def __init__(self, in_dim, hid):
        super().__init__()
        self.gru = nn.GRUCell(in_dim, hid)
        self.w = nn.Parameter(torch.zeros(hid))   # per-unit decay rate
        self.b = nn.Parameter(torch.zeros(hid))

    def forward(self, x_t, dt_t, h):
        # dt_t: (batch, 1) elapsed time since previous sample, in seconds
        gamma = torch.exp(-torch.relu(self.w * dt_t + self.b))  # (batch, hid)
        h_decayed = gamma * h                                   # forget by gap
        return self.gru(x_t, h_decayed)

cell = GRUDecayCell(in_dim=4, hid=16)
h = torch.zeros(2, 16)
# two irregular samples: a 0.1 s gap, then a 30 s gap
for x, dt in [(torch.randn(2, 4), torch.full((2, 1), 0.1)),
              (torch.randn(2, 4), torch.full((2, 1), 30.0))]:
    h = cell(x, dt, h)
print(h.shape)  # torch.Size([2, 16])

The code above shows the decay applied to two samples spaced 0.1 s and 30 s apart; the second update starts from a heavily faded memory, which is the whole point.

Asynchronous multi-sensor streams, and the leakage traps

When several channels each fire on their own clock, you have two structural choices. Union-of-timestamps builds one merged sequence over every arrival time across all sensors; at each row, exactly one channel (or a few) is real and the rest are masked, and per-channel \(\Delta t\) tells the model how stale each masked entry is. This preserves the true event order and is the natural input for a decay or ODE model. Set-based encoders drop the grid entirely: the record becomes an unordered set of \((t, \text{channel}, \text{value})\) tuples fed to a permutation-invariant model. This works because self-attention scores every pair of tokens by content and position directly, so nothing forces the input onto a shared clock. Each token's position is a continuous encoding of its raw timestamp, typically a sinusoidal or learned embedding in the style of Time2Vec, rather than an integer index, so a 50 ms gap and a 5 s gap remain distinguishable to the attention weights even though both pairs sit next to each other in the tuple list. This is the door into the attention-based sensor models of Chapter 15, and it earns its cost when channels are numerous and sparse enough that no shared grid is natural, dozens of independently ordered clinical labs, for instance, rather than the two- or three-sensor case where a union-of-timestamps table suffices.

Two evaluation traps deserve a warning label. First, any imputation statistic, a channel mean, a decay baseline, a normalization scale, must be estimated on the training split only, or the test set's future bleeds into training exactly as it would for the continuous-health monitors of Chapter 33. Second, and subtler: if sampling times themselves correlate with the label (a lab ordered because the clinician already suspected the outcome), the timestamp pattern can be a shortcut feature absent at prediction time. Audit whether \(\Delta t\) is a legitimate predictor or a leak of the decision you are trying to anticipate.

What's Next

In Section 14.7, we assemble everything from this chapter, the recurrent and convolutional cores, the streaming state, and the irregular-time inputs, into practical training recipes: how to window, normalize, batch ragged sequences, schedule learning rates, and regularize sensor models so they actually train stably and survive contact with a live deployment.