Appendix E. Sensor Datasets and Benchmarks

This appendix is a working reference for the datasets that recur throughout the book. It is organized by modality and task, and every entry names the pitfall that most often invalidates a result on that dataset. Sensor data has a habit of leaking: a naive random split lets the same subject, the same machine, or the same drive appear in both training and test, and the reported number then measures memorization instead of generalization. Read the Notes column before you read anyone's leaderboard.

How to read the pitfalls

Three failure modes dominate. Subject leakage occurs when windows from one person land on both sides of the split, so the model recognizes the person rather than the activity. Device or machine leakage is the industrial analogue: the same bearing, pump, or building appears in train and test, and the model keys on a serial-number fingerprint. Domain shift is the honest hardness that remains once leakage is removed: a new sensor placement, a new city, a new operating regime. A good benchmark defines the split so that the intended shift is the only thing the test measures. When it does not, you must define it yourself.

Curated datasets by modality and task

A note on leakage-safe splitting

The single rule that prevents most of the failures above is to split on the grouping variable, not on the window. Let \(g(x)\) be the group that a sample belongs to: the subject for HAR and biosignals, the machine or unit for industrial data, the drive or scene for autonomous data, and the calendar boundary for forecasting. A split is leakage-safe when no group straddles the boundary, that is when the train and test group sets are disjoint:

In practice you hand the grouping vector to a group-aware splitter and never touch the raw window index. The snippet below shows the pattern for a subject-grouped HAR set; the same call works for machine IDs, scene tokens, or any other group.

from sklearn.model_selection import GroupShuffleSplit

# X: windows, y: labels, groups: subject id per window
splitter = GroupShuffleSplit(n_splits=1, test_size=0.2, random_state=0)
train_idx, test_idx = next(splitter.split(X, y, groups=subject_ids))

# Guarantee: no subject appears on both sides
assert set(subject_ids[train_idx]).isdisjoint(subject_ids[test_idx])

Group-aware splitting for HAR. Swap subject_ids for machine, drive, or scene identifiers to reuse the pattern across modalities.

Two boundary cases deserve care. For time series, grouping is not enough on its own: even within a single test series, an evaluation window whose input range overlaps the training range leaks the future, so forecasting splits must be strictly chronological with a gap at least as long as the forecast horizon. For anomaly and intrusion datasets such as SWaT and MIMII, the training set is normal-only by construction, so the risk shifts from split leakage to metric inflation; prefer event-level or range-based scoring over point-adjusted accuracy, which can turn a single lucky detection into a near-perfect number. When a benchmark ships an official split, use it, both because it encodes the intended domain shift and because it is the only way your numbers are comparable to everyone else's.