This module gathers the mathematical tools the rest of the course leans on: a little linear algebra, the language of expectation and covariance, the multivariate Gaussian, and the four probability quantities (likelihood, prior, posterior, evidence) that the next module turns into a way of reasoning. It is a reference to return to, not a full treatment.

- Recall the vector and matrix operations used throughout: dot product, matrix-vector product, transpose, inverse, and norm.

- Define expectation, variance, and covariance, and assemble the covariance matrix.

- Write the multivariate Gaussian density and read its shape from the covariance.

- Name the likelihood, prior, posterior, and evidence, and relate them by Bayes' rule.

A feature vector lives in $\mathbb{R}^n$ and a dataset stacks such vectors into a matrix. The dot product of two vectors sums their elementwise products:

$$\boxed{ x^T y = \sum_{i=1}^{n} x_i\, y_i }$$

A matrix $A$ maps a vector by the matrix-vector product $Ax$, the transpose $A^T$ swaps rows and columns, and the inverse $A^{-1}$ (when it exists) undoes $A$, so $A^{-1}A = I$. The Euclidean norm measures length:

$$\boxed{ \lVert x \rVert_2 = \sqrt{x^T x} }$$

A square matrix is symmetric if $A = A^T$, and positive semidefinite if $x^T A x \ge 0$ for every $x$. Covariance matrices, which appear next, are always symmetric and positive semidefinite.

The expectation is the probability-weighted average of a random variable, a sum in the discrete case and an integral in the continuous one:

$$\boxed{ \mathbb{E}[X] = \sum_x x\, p(x) \qquad \mathbb{E}[X] = \int x\, p(x)\, dx }$$

Expectation is linear, $\mathbb{E}[aX + b] = a\,\mathbb{E}[X] + b$. The variance measures spread around the mean $\mu = \mathbb{E}[X]$:

$$\boxed{ \mathrm{Var}(X) = \mathbb{E}\!\left[(X - \mu)^2\right] = \mathbb{E}[X^2] - \mu^2 }$$

Covariance measures how two variables move together:

$$\boxed{ \mathrm{Cov}(X, Y) = \mathbb{E}\!\left[(X - \mu_X)(Y - \mu_Y)\right] }$$

For a random vector $x \in \mathbb{R}^n$ with mean $\mu$, the covariance matrix collects every pairwise covariance:

$$\boxed{ \Sigma = \mathbb{E}\!\left[(x - \mu)(x - \mu)^T\right], \qquad \Sigma_{ij} = \mathrm{Cov}(x_i, x_j) }$$

Its diagonal holds the per-feature variances, it is symmetric, and it is positive semidefinite. Off-diagonal entries record correlation between features.

The Gaussian is the default model for continuous noise and for smooth clouds of points. In $n$ dimensions it is parameterized by a mean vector $\mu$ and a covariance matrix $\Sigma$:

$$\boxed{ p(x) = \frac{1}{(2\pi)^{n/2}\,|\Sigma|^{1/2}} \exp\!\left(-\tfrac{1}{2}(x - \mu)^T \Sigma^{-1}(x - \mu)\right) }$$

Its contours of equal density are ellipsoids centred at $\mu$, and the covariance $\Sigma$ sets their spread and orientation.


*Remark:* the quadratic form $(x - \mu)^T \Sigma^{-1}(x - \mu)$ is the squared Mahalanobis distance, the natural distance once the data has a covariance structure.

Almost every model in this course reasons about parameters $\theta$ given data $D$. Four quantities recur, and they are tied together by Bayes' rule:

$$\boxed{ p(\theta \mid D) = \frac{p(D \mid \theta)\, p(\theta)}{p(D)} }$$

- The **likelihood** $p(D \mid \theta)$ is how probable the data is under a given $\theta$.

- The **prior** $p(\theta)$ is what we believed about $\theta$ before seeing the data.

- The **posterior** $p(\theta \mid D)$ is the updated belief after seeing it.

- The **evidence** $p(D) = \int p(D \mid \theta)\, p(\theta)\, d\theta$ normalizes the posterior so it integrates to one.


*Remark:* the evidence is a constant with respect to $\theta$, so for many tasks it can be ignored and only the numerator $p(D \mid \theta)\, p(\theta)$ matters.

---

Next: [Course overview](/en/Mathematics)