This appendix is a working shelf, not a survey. Every entry is a library you can install today and point at a real signal: an accelerometer trace, a LiDAR sweep, an EEG montage, a radar cube. The groups follow the arc of a sensory AI project, from loading raw arrays to shipping a quantized model onto a microcontroller. Pick one tool per row, learn its data model deeply, and treat the rest as fallbacks. A stack of six libraries you understand beats twenty you have merely imported.
Core numerics and deep learning
The foundation is shared with all of machine learning, so we keep it short. These are the packages every later group imports.
| Tool | Purpose |
|---|---|
| Python | The glue language for the entire stack. |
| NumPy | The n-dimensional array that every sensor buffer becomes. |
| SciPy | Filtering, FFTs, interpolation, and classical signal processing. |
| pandas | Labeled tabular and time-indexed data wrangling. |
| Polars | A faster, multi-threaded DataFrame for large logs. |
| PyTorch | The default deep learning framework in this book. |
| PyTorch Lightning | Training-loop structure without boilerplate. |
| scikit-learn | Classical models, pipelines, and honest baselines. |
Time series
Most sensor streams are time series before they are anything else. These libraries give you windowing, forecasting, and classification with a consistent API.
- sktime: unified interface for forecasting, classification, and transformation.
- aeon: modern time-series ML with a large classifier and clustering zoo.
- tsai: deep-learning time-series models on top of PyTorch and fastai.
- GluonTS: probabilistic forecasting with deep and structural models.
- Nixtla: fast statistical, ML, and neural forecasting (StatsForecast, NeuralForecast).
State estimation
When a signal reflects a hidden physical state, you estimate rather than classify. These cover the recursive filters and factor graphs of Part on fusion.
- filterpy: Kalman, extended, and unscented filters with readable source.
- GTSAM: factor-graph smoothing and optimization for SLAM and sensor fusion.
Foundation models and self-supervised learning
Pretrained backbones now exist for time series, not just text and images. Reach for these when labels are scarce and the signal family is common.
- Hugging Face: the hub and Transformers library for pretrained models.
- uni2ts: the Moirai family of universal time-series transformers.
- granite-tsfm: IBM Granite Tiny Time Mixers, small and fast forecasters.
- momentfm: MOMENT foundation models for forecasting, imputation, and anomaly work.
- Chronos: tokenized time series treated as a language-model problem.
- mamba-ssm: selective state-space sequence models for long signals.
3D and perception
Point clouds, meshes, and radiance fields need their own machinery. These handle geometry, detection, and neural scene representation.
- Open3D: point cloud and mesh processing, registration, and visualization.
- MMDetection3D: LiDAR and multi-modal 3D object detection.
- OpenPCDet: focused LiDAR detection with strong reference models.
- Pointcept: point-cloud representation learning and segmentation.
- gsplat: differentiable 3D Gaussian splatting.
- nerfstudio: a modular workbench for neural radiance fields.
Biosignals
ECG, EEG, EMG, PPG, and respiration carry their own conventions and file formats. These libraries encode decades of clinical practice.
- NeuroKit2: high-level processing for ECG, EDA, EMG, PPG, and respiration.
- MNE-Python: the standard for EEG and MEG analysis.
- Braindecode: deep learning on raw EEG with PyTorch.
- WFDB: read and write PhysioNet waveform records.
Radar and event-based sensing
Non-optical and neuromorphic sensors reward specialized tooling for their unusual data shapes.
- OpenRadar: processing pipelines for mmWave FMCW radar.
- Metavision and Tonic: event-camera SDKs and dataset tooling for asynchronous events.
- snnTorch and Lava: spiking neural networks for neuromorphic hardware.
Robotics and simulation
Synthetic sensors let you pretrain and stress-test before touching hardware.
- ROS 2: middleware, message types, and drivers for real robots.
- Isaac Sim: photoreal simulation with physically based sensor models.
- CARLA: open urban driving simulator with camera, LiDAR, and radar.
- MuJoCo: fast contact-rich physics for control and proprioception.
Edge deployment
A sensory model that cannot run at the sensor is only half finished. These target laptops down to microcontrollers.
- ONNX Runtime: portable, accelerated inference across platforms.
- LiteRT and TFLite Micro: on-device inference, down to kilobyte-scale MCUs.
- ExecuTorch: PyTorch inference for mobile and embedded targets.
- microTVM: compile models to bare-metal devices.
- Edge Impulse: an end-to-end pipeline for embedded sensor ML.
Uncertainty and evaluation
Deployed sensory systems must know when to abstain. These add calibrated confidence, conformal guarantees, and out-of-distribution detection.
- MAPIE: conformal prediction sets and intervals for scikit-learn models.
- TorchCP: conformal prediction for deep models in PyTorch.
- TSInterpret: interpretability for time-series classifiers.
- OpenOOD: standardized out-of-distribution detection benchmarks.
Operations and serving
The last group turns experiments into systems: tracking, federated and streaming learning, and serving.
- MLflow: experiment tracking, model registry, and packaging.
- Weights & Biases: experiment logging, sweeps, and dashboards.
- Flower: federated learning across distributed sensor fleets.
- River: online learning on unbounded streams.
- FastAPI: a typed HTTP layer for model endpoints.
- Docker: reproducible environments from laptop to cloud.
A minimal project touches one tool from most groups. A gesture recognizer, for example, might load an IMU log with pandas, window it with aeon, fine-tune a tsai model, calibrate it with MAPIE, export through ExecuTorch, and track the whole run in MLflow. Start narrow, measure honestly, and add a library only when the signal itself demands it.