Types of Machine Learning

Lecture 1.2 introduced machine learning through the T/P/E framework. This lecture zooms in on the three major paradigms — supervised, unsupervised, and reinforcement learning — and adds a fourth, semi-supervised learning, that sits between the first two.

1. The Three Paradigms

The key distinguishing factor across paradigms is the form of the experience $E$:

• Supervised learning: experience comes as input–target pairs $(\mathbf{x}_i, t_i)$. Both inputs and labels are provided upfront.
• Unsupervised learning: experience is inputs only $\{\mathbf{x}_i\}$ — no labels. The algorithm discovers structure in the data.
• Reinforcement learning: experience is not provided upfront. The agent collects it by interacting with an environment — trial and error.

2. Supervised Learning

Supervised methods always operate on paired data. What differs between the two main subtypes is the nature of the target $t$.

In both cases the objective is the same: find a function $f$ that maps inputs to targets as accurately as possible, not just on the training data but on unseen data. That ability to perform well on new inputs is generalization. As we saw in lecture 1.2, a model that overfits the training set fails to generalize — it has learned the noise, not the signal.

3. Unsupervised Learning

Without labels the task changes: we look for hidden structure in the data. Two important examples are compression and clustering.

3.1 Compression via PCA

Imagine storing thumbnail images (100 × 100 pixels = 10,000 values per image) for millions of users. Storage is expensive; we want to represent each image with far fewer numbers without losing too much quality.

Principal Component Analysis (PCA) achieves this by exploiting shared structure across images. Applied to a dataset of face images, PCA computes:

• The mean image $\bar{\mathbf{x}}$ — a smooth, generic face that captures what all images have in common.
• A set of principal components $\mathbf{u}_1, \mathbf{u}_2, \ldots$ (also called eigenfaces in computer vision) — directions in pixel space that capture the most common variations: lighting, expression, hair, skin tone, etc.

Any face image can then be approximated by adding a weighted combination of these components to the mean: $$ \mathbf{x} \approx \bar{\mathbf{x}} + \sum_{i=1}^{M} \alpha_i \mathbf{u}_i $$ The coefficients $\alpha_1, \ldots, \alpha_M$ are all that need to be stored — just $M$ numbers instead of 10,000 pixels.

3.2 Clustering

Clustering groups data points by similarity without any labels — we already covered this in lecture 1.2 with the tumor gene-expression example. The key idea: by partitioning samples into clusters and computing cluster means, we can discover natural categories in the data and use them to inform decisions (e.g. selecting a treatment protocol for a new patient based on which cluster their tumor profile falls into).

4. Semi-Supervised Learning

A practical middle ground between supervised and unsupervised learning: we have $N$ inputs $\{\mathbf{x}_1, \ldots, \mathbf{x}_N\}$ but labels for only a subset $\{t_1, \ldots, t_k\}$, where $k \ll N$. The goal is to exploit all available data — labeled and unlabeled — rather than discarding the unlabeled portion.

5. Reinforcement Learning

Reinforcement learning (RL) is conceptually distinct from all the above. There is no fixed dataset. Instead, an agent operates in an environment and learns by taking actions and observing outcomes.

RL thrives in virtual environments because mistakes are cheap: a bad move in a simulated game costs nothing. This is why most successful RL applications are in games (Chess, Go, Starcraft) and simulation-based robotics. Deploying RL in the physical world is harder — mistakes can be costly or dangerous, and the real world cannot be simulated perfectly.

6. Summary