# 1. Mathematical refresher This module gathers the mathematical tools the rest of the course leans on: a little linear algebra, the language of expectation and covariance, the multivariate Gaussian, and the four probability quantities (likelihood, prior, posterior, evidence) that the next module turns into a way of reasoning. It is a reference to return to, not a full treatment. ## 1.1 Linear algebra A feature vector lives in $\mathbb{R}^n$ and a dataset stacks such vectors into a matrix. The dot product of two vectors sums their elementwise products: $$\boxed{ x^T y = \sum_{i=1}^{n} x_i\, y_i }$$ A matrix $A$ maps a vector by the matrix-vector product $Ax$, the transpose $A^T$ swaps rows and columns, and the inverse $A^{-1}$ (when it exists) undoes $A$, so $A^{-1}A = I$. The Euclidean norm measures length: $$\boxed{ \lVert x \rVert_2 = \sqrt{x^T x} }$$ A square matrix is symmetric if $A = A^T$, and positive semidefinite if $x^T A x \ge 0$ for every $x$. Covariance matrices, which appear next, are always symmetric and positive semidefinite. ## 1.2 Expectation and variance The expectation is the probability-weighted average of a random variable, a sum in the discrete case and an integral in the continuous one: $$\boxed{ \mathbb{E}[X] = \sum_x x\, p(x) \qquad \mathbb{E}[X] = \int x\, p(x)\, dx }$$ Expectation is linear, $\mathbb{E}[aX + b] = a\,\mathbb{E}[X] + b$. The variance measures spread around the mean $\mu = \mathbb{E}[X]$: $$\boxed{ \mathrm{Var}(X) = \mathbb{E}\!\left[(X - \mu)^2\right] = \mathbb{E}[X^2] - \mu^2 }$$ ## 1.3 Covariance and the covariance matrix Covariance measures how two variables move together: $$\boxed{ \mathrm{Cov}(X, Y) = \mathbb{E}\!\left[(X - \mu_X)(Y - \mu_Y)\right] }$$ For a random vector $x \in \mathbb{R}^n$ with mean $\mu$, the covariance matrix collects every pairwise covariance: $$\boxed{ \Sigma = \mathbb{E}\!\left[(x - \mu)(x - \mu)^T\right], \qquad \Sigma_{ij} = \mathrm{Cov}(x_i, x_j) }$$ Its diagonal holds the per-feature variances, it is symmetric, and it is positive semidefinite. Off-diagonal entries record correlation between features. ## 1.4 The multivariate Gaussian The Gaussian is the default model for continuous noise and for smooth clouds of points. In $n$ dimensions it is parameterized by a mean vector $\mu$ and a covariance matrix $\Sigma$: $$\boxed{ p(x) = \frac{1}{(2\pi)^{n/2}\,|\Sigma|^{1/2}} \exp\!\left(-\tfrac{1}{2}(x - \mu)^T \Sigma^{-1}(x - \mu)\right) }$$ Its contours of equal density are ellipsoids centred at $\mu$, and the covariance $\Sigma$ sets their spread and orientation.  *A spherical covariance gives circular contours, a diagonal one gives axis-aligned ellipses, and off-diagonal terms tilt them, encoding correlation between the features.* *Remark:* the quadratic form $(x - \mu)^T \Sigma^{-1}(x - \mu)$ is the squared Mahalanobis distance, the natural distance once the data has a covariance structure. ## 1.5 Likelihood, prior, posterior, and evidence Almost every model in this course reasons about parameters $\theta$ given data $D$. Four quantities recur, and they are tied together by Bayes' rule: $$\boxed{ p(\theta \mid D) = \frac{p(D \mid \theta)\, p(\theta)}{p(D)} }$$ - The **likelihood** $p(D \mid \theta)$ is how probable the data is under a given $\theta$. - The **prior** $p(\theta)$ is what we believed about $\theta$ before seeing the data. - The **posterior** $p(\theta \mid D)$ is the updated belief after seeing it. - The **evidence** $p(D) = \int p(D \mid \theta)\, p(\theta)\, d\theta$ normalizes the posterior so it integrates to one.  *The posterior is proportional to the likelihood times the prior, divided by the evidence that makes it a proper distribution.* *Remark:* the evidence is a constant with respect to $\theta$, so for many tasks it can be ignored and only the numerator $p(D \mid \theta)\, p(\theta)$ matters. *These tools underpin the [Machine Learning](/en/Machine%20Learning) course, where probability, loss functions, and models are built on them.* --- Next: [Course overview](/en/Mathematics)