Semi-supervised learning: a few labels plus many unlabeled examples
Semi-supervised learning trains models with both labeled and unlabeled data, reducing the need for expensive annotation. It is used when labeled examples are scarce but unlabeled ones are plentiful. Techniques include self-training and label propagation.
Definition and Motivation
Semi-supervised learning is a machine learning approach that uses both labeled and unlabeled data to train models for classification and regression, as described by IBM's semi-supervised learning overview.
It is especially useful when labeled data is difficult or expensive to obtain but large amounts of unlabeled data are readily available. The technique reduces the annotation burden while still producing models with strong predictive performance.
Self-Training and Label Propagation
Common techniques include self-training, where a classifier is iteratively trained and adds confident predictions from unlabeled data, and label propagation, which builds a similarity graph to spread labels. The scikit-learn user guide details these methods, including the use of -1 to mark unlabeled samples and the availability of LabelPropagation and LabelSpreading variants (scikit-learn's semi-supervised guide).
These algorithms rely on assumptions about the data distribution, such as smoothness or cluster structure, to effectively use the unlabeled examples. In self-training, the most confident predictions are added per iteration; label propagation spreads labels along graph edges.
Supervised, Unsupervised, and Semi-supervised
In supervised learning, models are trained exclusively on labeled data, where each input has a ground truth output; the loss function measures prediction error during training, as explained in IBM's guide to supervised learning.
Unsupervised learning, by contrast, uses no labels and aims to discover hidden structures or clusters in the data, according to IBM's overview of unsupervised learning. Semi-supervised learning combines both, using the limited labeled points to ground the model and the abundant unlabeled points to inform the data distribution.
The Cost of Data Labeling
Data labeling is a significant bottleneck in machine learning because it requires human effort to annotate raw data such as images, text, or videos. IBM's data labeling guide reports that labeled data is more difficult and expensive to acquire than unlabeled data, and it discusses approaches like internal labeling, outsourcing, and crowdsourcing, each with its own trade-offs (IBM's data labeling guide).
Semi-supervised learning directly addresses this challenge by reducing the amount of labeled data needed, allowing organizations to take advantage of the vast amounts of unlabeled data they may already have.
Evaluation and Practical Considerations
When evaluating any classification model, accuracy is a basic metric defined as the number of correct predictions divided by total predictions, but it can be misleading for imbalanced datasets, as noted in Google's machine learning glossary.
For semi-supervised models, it is important to verify that the added unlabeled data actually improves generalization and does not introduce bias, since the algorithms rely on assumptions about the data distribution. Validation with a held-out labeled test set helps measure the true performance.
| Paradigm | Data used | Training signal | Typical tasks |
|---|---|---|---|
| Supervised | Labeled data only | Ground truth labels | Classification, regression |
| Unsupervised | Unlabeled data only | None (finds patterns) | Clustering, association, dimensionality reduction |
| Semi-supervised | Both labeled and unlabeled | Labels plus data distribution | When labels are scarce, e.g., image classification |
Frequently asked questions
What is semi-supervised learning?
Semi-supervised learning uses both labeled and unlabeled data to train models, combining elements of supervised and unsupervised learning.
When is semi-supervised learning useful?
When labeled data is scarce or expensive to obtain but unlabeled data is abundant, such as in image segmentation or natural language processing.
How does self-training work?
A base classifier is trained on labeled data, then predicts labels for unlabeled data; the most confident predictions are added to the training set, and the process repeats.
What is label propagation?
Label propagation builds a similarity graph over all samples and spreads class labels from labeled to unlabeled points based on graph structure.
What are the limitations of semi-supervised learning?
The algorithms rely on distributional assumptions; if those assumptions do not hold, the unlabeled data may hurt performance.