Reproducible, versioned sensor pipelines and data contracts

"A result you cannot reproduce is a rumor with a confidence interval."

A Reproducible AI Agent

Six weeks after the results were signed off, a model's accuracy quietly slid two points, and nothing in the code had changed: a resampler had picked up a new library default, and every training window was now subtly different from the one that produced the reported number. Stopping that silent rewrite is the whole job of this section, and it builds directly on machinery you already have: the sensor formats and timestamping of Chapter 3 and the windowing, split, and calibration choices of Sections 5.2 through 5.6, which we now treat as a single chain to be versioned. This section stays clear of Chapter 69 (operating deployed models across a fleet) and Chapter 65 (evaluation protocols); here the deliverable is a dataset build that anyone can regenerate byte-for-byte and trust on arrival.

Why sensor pipelines rot, and what reproducibility actually requires

What. A sensor pipeline is the ordered sequence of transformations that turns raw device output into model-ready tensors: decode, resample, synchronize, filter, window, normalize, label, split. Why it rots. Every stage hides a moving part. A resampler pulls in a new SciPy release with a different default; a firmware update changes the accelerometer full-scale range; someone re-runs the split with a fresh random seed; a timezone assumption flips at a daylight-saving boundary. None of these throw an error. They just quietly change the dataset, and six weeks later a model that "regressed" is really a model trained on different data.

How reproducibility is achieved. That quiet two-point slide could just as easily have been a failed regulatory audit or a wearable recalled from thousands of wrists, and by the time anyone notices, the raw evidence needed to explain it is often already overwritten. The only real defense is to make the build itself refuse to change without saying so. Determinism is not one thing but four, and you need all four to actually hold. First, fix the inputs: content-address the raw data so a hash identifies the exact bytes, not a mutable path. Second, fix the code: pin the pipeline to a commit and pin every library version. Third, fix the configuration: put window length, stride, seed, filter cutoffs, and split policy in a versioned config file, never in a notebook cell. Fourth, fix the environment and randomness: seed every random operation and record the interpreter and operating system (OS). The output identity you want is a function of all four:

where \(H\) is a cryptographic hash. If any input changes, the id changes, and you have caught a silent mutation before it reached a model. If all four match, re-running the build reproduces the same tensors in practice, which is the whole game. In short: version the recipe, not the result, and every unrequested change is forced to announce itself as a changed id. Figure 5.7.1 shows the four pinned inputs converging through the hash into a single identity that any consumer can check on arrival.

Data contracts: promises the pipeline must keep

A versioned pipeline guarantees that the same bytes come out every run, but it says nothing about whether those bytes are physically sensible; that gap is exactly what a data contract closes. What. A data contract is an explicit, machine-checkable specification of what a dataset or a pipeline stage guarantees: the schema (channel names, dtypes, units), the physical ranges, the sampling rate and its allowed jitter, the null and gap policy, and the label vocabulary. It is the application programming interface (API) of your data. Why. Without a contract, every consumer re-discovers the data's quirks by trial and error, and a producer can break a downstream model just by shipping a new sensor batch in millivolts instead of volts. A contract moves that failure to the boundary where it belongs: the moment bad data arrives, a validation gate rejects it with a specific reason, instead of a model degrading mysteriously a month later.

How. A contract has two halves. The declarative schema lists fields, types, and units and is checked cheaply on every batch. The statistical expectations assert distributional facts: the accelerometer magnitude at rest sits near \(9.81\ \mathrm{m/s^2}\), the fraction of missing samples per window stays below a threshold, the inter-sample interval clusters at \(\Delta t = 1/f_s\). These expectations connect directly to the sensor physics of Chapter 2: a contract that knows gravity's magnitude can flag a mislabeled axis or a wrong full-scale setting automatically. When. Enforce the contract at three gates: at ingestion (raw data crossing into your system), between stages (each transform validates its output), and at the training boundary (the final tensor matches what the model expects). Contracts should fail closed: a violation halts the build rather than passing suspect data through.

Building a versioned pipeline in practice

Knowing what a contract must promise tells you nothing about the machinery that enforces it every run; that machinery is the versioned pipeline itself. How to structure it. Express the pipeline as a directed acyclic graph (DAG) of stages, each a pure function from input artifact to output artifact, tagged with the hash of its inputs and config. Two properties follow for free: caching, since a stage whose input hash is unchanged is skipped, and lineage, since every tensor can name the raw file, commit, and config that produced it. A reviewer asking where a training example came from gets a lookup, not an archaeology project. When to version what. Version the raw data immutably by content address, the code by git commit, the config in the repo, and the dataset as a manifest (a small metadata file that records the four-part id and the inputs it came from). Never version derived tensors by copying them around; version the recipe and cache its output.

A DAG is a set of stages joined by dependency edges that never form a cycle. Because no edge cycles back, there is always a well-defined order in which to run the stages, and no stage can depend, directly or transitively, on its own output. That acyclic structure is exactly what lets the build compute each stage's input hash before it runs, which is what makes caching and lineage possible at all. The build walks the graph in topological order, meaning every stage is visited only after all the stages it depends on, and skips any stage whose combined input-and-config hash matches the previous run. Reach for a DAG runner instead of a linear script the moment stages start sharing intermediate artifacts or you want partial rebuilds; a plain top-to-bottom script is fine only while the pipeline is a single unbranched chain. Figure 5.7.2 illustrates a versioned pipeline DAG with per-stage hashing, caching, and lineage.

The snippet below computes the dataset identity from its four inputs and writes a manifest. It is deliberately small; the point is that the identity is a deterministic function of pinned inputs, so two people running it on the same commit and config get the same id and can prove they built the same data.

import hashlib, json, subprocess, sys, platform
from pathlib import Path

def sha256_file(path, chunk=1 << 20):
    h = hashlib.sha256()
    with open(path, "rb") as f:
        for block in iter(lambda: f.read(chunk), b""):
            h.update(block)
    return h.hexdigest()

def hash_str(s):
    return hashlib.sha256(s.encode()).hexdigest()

def build_manifest(raw_files, config):
    raw_hash   = hash_str("".join(sorted(sha256_file(p) for p in raw_files)))
    code_commit = subprocess.check_output(
        ["git", "rev-parse", "HEAD"]).decode().strip()
    config_hash = hash_str(json.dumps(config, sort_keys=True))
    env_hash    = hash_str(f"{sys.version}|{platform.platform()}")
    dataset_id  = hash_str(raw_hash + code_commit + config_hash + env_hash)[:16]
    return {
        "dataset_id": dataset_id,
        "raw_hash": raw_hash, "code_commit": code_commit,
        "config_hash": config_hash, "env_hash": env_hash,
        "config": config,
    }

if __name__ == "__main__":
    cfg = {"window_len": 256, "stride": 64, "fs_hz": 50,
           "split": "subject_disjoint", "seed": 7}
    manifest = build_manifest(sorted(Path("raw").glob("*.parquet")), cfg)
    Path("dataset_manifest.json").write_text(json.dumps(manifest, indent=2))
    print("dataset_id:", manifest["dataset_id"])

Note the two subtle correctness moves in the code. Raw hashes are sorted before concatenation so file order does not affect the id, and the config is serialized with sort_keys=True so a reordered dictionary still hashes identically. Both prevent spurious id changes that would train teams to ignore the alarm. This same discipline underpins the leakage-safe benchmarking of Chapter 65: a benchmark is only fair if everyone provably built the same splits.

When reproducibility is worth the cost, and when it is not

Now that you can build a fully versioned pipeline, the harder question is when the machinery earns its keep, because all of this determinism is not free. What it costs. Full versioning adds friction: hashing large raw archives takes time, you must rebuild pinned environments, and a contract that fails closed will occasionally block a build at an inconvenient hour. When to pay it. Pay in full whenever you will publish, audit, or ship a result to a device you cannot easily update, and whenever more than one person touches the pipeline. A clinical study, a regulated wearable (the validation regime of Chapter 34), and any leaderboard submission all demand it, because the question "is this the same data" must have a provable answer. When to lighten up. A solo exploratory notebook that will be thrown away next week does not need a content-addressed remote; pin the seed and the library versions and move on. The skill is matching the ceremony to the stakes: enough determinism that you can trust and defend the number, not so much that exploration grinds to a halt.

What's Next

In Chapter 6, we open Part II and start turning clean, versioned streams into clean signals: moving averages, finite impulse response (FIR) and infinite impulse response (IIR) filters, and the band-pass and notch designs that strip noise while preserving the phenomenon. Every filter you meet there is a pipeline stage in the sense of this section, with a cutoff and a design choice that belongs in your versioned config and under a data contract; reproducibility is not left behind when we move from data engineering to signal processing, it comes along as a habit.