The capstone is where the whole book comes together. Every part you have worked through taught one of the six verbs of sensory AI: sense, acquire, represent, learn, evaluate, and deploy. In isolation each verb is a skill. The capstone asks you to run all six, in order, on a single system of your choice, and to defend the result the way an engineer defends a product review. You pick one application and one sensing stack, then carry it from raw physical signal to a deployed or simulated model backed by an honest report.
This is a synthesis project, not a new-topic project. You should reach for chapters you have already read rather than inventing fresh theory. The in-text chapter that frames the exercise and gives worked scaffolding is Chapter 72: Frontier Research and the End-to-End Capstone; read it alongside this page before you commit to an option.
The end-to-end arc
Whichever option you choose, the workflow is fixed. Follow these seven stages in sequence, because each one constrains the next.
- Select application and sensing stack. Name the physical quantity you are measuring, the transducer, the sampling rate, and the deployment target.
- Define measurement and task. State the learning task (classification, regression, detection, or forecasting), the label source, and the metric you will optimize. Write the operating point down before you see any results.
- Build acquisition and preprocessing. Assemble the data path: sampling, synchronization, filtering, windowing, calibration, and feature or spectrogram extraction.
- Train a classical baseline and a modern model. Fit a transparent baseline (for example a random forest on hand-designed features) and a modern or foundation model (a deep network, a pretrained audio or time-series encoder, or a fine-tuned foundation model). The baseline is not optional; it is your sanity floor.
- Evaluate generalization, uncertainty, and robustness. Use leakage-safe splits (subject-wise, session-wise, or time-blocked, never random row shuffling on windowed streams). Report calibrated uncertainty and degradation under shift and noise.
- Deploy or simulate deployment. Export the model, measure latency and memory on the target, and run a streaming inference loop on held-out data.
- Write the final report. Cover the technical system and a responsible-use section: failure modes, population coverage, privacy, and the cost of a wrong prediction.
A useful framing for stage 5 is the generalization gap. If \(R_{\text{emp}}\) is error on your training distribution and \(R_{\text{shift}}\) is error under a realistic deployment shift, then
$$\Delta = R_{\text{shift}} - R_{\text{emp}}$$is the number a reviewer will ask about first. A small \(\Delta\) with an honest split is worth more than a spectacular in-distribution score.
Capstone options
Pick exactly one. Each is scoped to a public or self-collected dataset and a plausible edge target.
| Option | Primary sensors | Core task |
|---|---|---|
| Wearable activity and anomaly monitoring | IMU, PPG, accelerometer | Activity classification plus rare-event anomaly detection |
| Industrial predictive maintenance | Vibration, current, acoustic | Fault classification and remaining-useful-life regression |
| Smart-building occupancy | CO2, PIR, temperature, humidity | Occupancy count estimation from ambient signals |
| Radar, lidar, or depth perception prototype | FMCW radar, lidar, or depth camera | Presence, range, and motion classification |
| Multi-sensor mobile-robot fusion | IMU, wheel odometry, lidar, camera | State estimation and terrain or obstacle classification |
| Environmental early-warning network | Air quality, seismic, or hydrological arrays | Threshold-crossing forecasting and event detection |
| Sensor-language monitoring agent | Any stream plus a language model | Natural-language alerts and query answering over sensor state |
Deliverables
Submit four artifacts: (1) a reproducible code repository with a one-command data-to-metrics pipeline and a fixed random seed; (2) a data card describing provenance, splits, and known biases; (3) a model card that places the baseline and the modern model side by side on the same split and metric; and (4) a technical and responsible-use report of roughly 3000 to 5000 words with figures.
Rubric checklist
| Criterion | What passes |
|---|---|
| Task definition | Metric and operating point fixed before results are seen |
| Leakage safety | Splits respect subject, session, or time; no window bleed between train and test |
| Baseline present | Classical model reported on the identical split and metric as the modern model |
| Generalization | Performance under at least one realistic shift is measured, not assumed |
| Uncertainty | Calibration or confidence intervals reported, not point scores alone |
| Deployment | Latency, memory, and streaming behavior measured on the target |
| Responsible use | Failure modes, coverage gaps, and privacy addressed concretely |
| Reproducibility | Repository runs end to end from a clean checkout |
Treat the checklist as a gate, not a bonus sheet. A model that scores well but leaks across the split fails the capstone, because it would fail in the field. When you finish, you will have done in miniature what a sensory-AI team does for real: turned a physical measurement into a trustworthy decision, and written down why anyone should believe it.