This appendix is a compact refresher for the mathematics that recurs throughout the book. It is not a substitute for a course; it collects the definitions and formulas you will reach for while reading, so you do not have to leave the page to recall what a Nyquist rate or a Kalman gain is. Each section gives a tight statement plus the key equations.
Sampling and the Nyquist Theorem
A continuous signal \(x(t)\) becomes a discrete sequence by sampling at a fixed interval \(T_s\), giving samples \(x[n] = x(nT_s)\) taken at rate \(f_s = 1/T_s\). The Nyquist-Shannon sampling theorem states that a signal whose spectrum is zero above a bandwidth \(B\) is fully reconstructable from its samples if and only if it is sampled faster than twice that bandwidth. Sampling too slowly folds high frequencies down into low ones, an irreversible corruption called aliasing, which is why sensor front ends place an analog low-pass (anti-aliasing) filter before the analog-to-digital converter.
\[ f_s > 2B, \qquad f_{\text{Nyquist}} = \frac{f_s}{2} \]The Fourier Transform and the DFT/FFT
The Fourier transform expresses a signal as a sum of complex exponentials, exposing which frequencies it contains. For continuous signals the transform and its inverse are integrals; for a length-\(N\) sampled block the discrete Fourier transform (DFT) is a finite sum. The fast Fourier transform (FFT) is not a different transform but an algorithm that computes the DFT in \(O(N \log N)\) operations instead of \(O(N^2)\), which is what makes real-time spectral analysis on sensor streams practical.
\[ X(f) = \int_{-\infty}^{\infty} x(t)\, e^{-i 2\pi f t}\, dt \qquad X[k] = \sum_{n=0}^{N-1} x[n]\, e^{-i 2\pi k n / N} \]The bin index \(k\) corresponds to physical frequency \(f_k = k f_s / N\), so frequency resolution improves with a longer analysis window.
Convolution and LTI Filters
A linear time-invariant (LTI) system is completely described by its impulse response \(h\): its output for any input is the convolution of the input with \(h\). Convolution in time corresponds to multiplication in frequency, so filtering is often analyzed as shaping the signal spectrum by the filter's frequency response \(H(f)\). This duality is the workhorse behind smoothing, differentiation, band-pass extraction, and the convolutional layers of neural networks.
\[ (x * h)[n] = \sum_{k=-\infty}^{\infty} x[k]\, h[n-k], \qquad \mathcal{F}\{x * h\} = X(f)\, H(f) \]Probability
A random variable \(X\) is described by its distribution. The expectation is its probability-weighted average and the variance measures spread around that average.
\[ \mathbb{E}[X] = \int x\, p(x)\, dx, \qquad \operatorname{Var}(X) = \mathbb{E}\big[(X - \mathbb{E}[X])^2\big] = \mathbb{E}[X^2] - \mathbb{E}[X]^2 \]The Gaussian (normal) distribution appears everywhere in sensing because sums of many small independent disturbances tend toward it (the central limit theorem), and it is the maximum-entropy distribution for a fixed mean and variance.
\[ p(x) = \frac{1}{\sqrt{2\pi\sigma^2}}\, \exp\!\left(-\frac{(x-\mu)^2}{2\sigma^2}\right) \]Bayes' rule inverts a conditional probability, turning a sensor model \(p(z \mid x)\) (how measurements arise from a state) into a belief \(p(x \mid z)\) about the state given the measurement. This is the formal backbone of all recursive estimation in the book.
\[ p(x \mid z) = \frac{p(z \mid x)\, p(x)}{p(z)} \]Estimation
Maximum likelihood estimation (MLE) picks the parameter that makes the observed data most probable; maximum a posteriori estimation (MAP) adds a prior, and reduces to MLE when the prior is flat. Working in log space turns products into sums and is numerically stable.
\[ \hat{\theta}_{\text{MLE}} = \arg\max_{\theta}\, \sum_i \log p(x_i \mid \theta), \qquad \hat{\theta}_{\text{MAP}} = \arg\max_{\theta}\, \Big[\log p(\theta) + \sum_i \log p(x_i \mid \theta)\Big] \]Any estimator's expected squared error decomposes into a bias term and a variance term. Simple models are biased but stable; flexible models reduce bias at the cost of variance. Choosing model capacity is choosing where to sit on this tradeoff.
\[ \mathbb{E}\big[(\hat{\theta} - \theta)^2\big] = \underbrace{\big(\mathbb{E}[\hat{\theta}] - \theta\big)^2}_{\text{bias}^2} + \underbrace{\operatorname{Var}(\hat{\theta})}_{\text{variance}} \]State-Space Models and the Kalman Filter
A linear-Gaussian state-space model describes a hidden state \(\mathbf{x}_k\) that evolves under a transition matrix \(\mathbf{F}\) with process noise \(\mathbf{w}\), observed through a measurement matrix \(\mathbf{H}\) with measurement noise \(\mathbf{v}\).
\[ \mathbf{x}_k = \mathbf{F}\mathbf{x}_{k-1} + \mathbf{w}_k, \qquad \mathbf{z}_k = \mathbf{H}\mathbf{x}_k + \mathbf{v}_k, \qquad \mathbf{w}_k \sim \mathcal{N}(0, \mathbf{Q}),\ \ \mathbf{v}_k \sim \mathcal{N}(0, \mathbf{R}) \]The Kalman filter is the optimal recursive estimator for this model. Each step predicts the state and its covariance forward, then corrects them using the new measurement, weighted by the Kalman gain \(\mathbf{K}_k\) that balances model confidence against measurement noise.
| Stage | Equations |
|---|---|
| Predict | \(\hat{\mathbf{x}}_k^- = \mathbf{F}\hat{\mathbf{x}}_{k-1}\), \(\mathbf{P}_k^- = \mathbf{F}\mathbf{P}_{k-1}\mathbf{F}^\top + \mathbf{Q}\) |
| Gain | \(\mathbf{K}_k = \mathbf{P}_k^-\mathbf{H}^\top\big(\mathbf{H}\mathbf{P}_k^-\mathbf{H}^\top + \mathbf{R}\big)^{-1}\) |
| Update | \(\hat{\mathbf{x}}_k = \hat{\mathbf{x}}_k^- + \mathbf{K}_k\big(\mathbf{z}_k - \mathbf{H}\hat{\mathbf{x}}_k^-\big)\), \(\mathbf{P}_k = (\mathbf{I} - \mathbf{K}_k\mathbf{H})\mathbf{P}_k^-\) |
When \(\mathbf{F}\) or \(\mathbf{H}\) is nonlinear, the extended and unscented variants linearize or sample around the current estimate, but the predict-correct skeleton is unchanged.
Basic Optimization: Gradient Descent
Most model fitting minimizes a loss \(L(\theta)\) by moving parameters against the gradient, the direction of steepest increase. The step size \(\eta\) (learning rate) sets how far each update travels: too large and it diverges, too small and it crawls. Stochastic gradient descent estimates the gradient from a small batch of samples per step, trading noise for speed and enabling training on data too large to process at once.
\[ \theta_{t+1} = \theta_t - \eta\, \nabla_\theta L(\theta_t) \]def gradient_descent(grad, theta, lr=0.01, steps=1000):
for _ in range(steps):
theta = theta - lr * grad(theta) # step opposite the gradient
return thetaThe full training loop in five lines: everything else (momentum, Adam, schedules) refines how the step is chosen.
Information Theory
Entropy measures the average uncertainty, in bits when the logarithm is base 2, of a random variable, and sets a floor on how compactly its outcomes can be encoded. A fair coin has one bit of entropy; a certain outcome has zero.
\[ H(X) = -\sum_x p(x)\, \log_2 p(x) \]Mutual information measures how much observing one variable reduces uncertainty about another. It is the natural score for how informative a sensor or a feature is about the quantity you care about, and it is zero exactly when the two are independent.
\[ I(X; Y) = \sum_{x,y} p(x,y)\, \log_2 \frac{p(x,y)}{p(x)\, p(y)} = H(X) - H(X \mid Y) \]Together these quantities let us reason about channel capacity, feature selection, and the fundamental limits on what any perception system can extract from noisy measurements.