Convolutional neural networks: filters, pooling and feature maps
Convolutional neural networks (CNNs) apply learnable filters to an input to create feature maps, which highlight useful patterns. Pooling layers reduce the size of these maps while preserving important information, and fully connected layers combine the features for classification. The design of filters, stride, padding, and pooling determines the network's trade-off between detail and efficiency.
Architecture overview
A convolutional neural network (CNN) is a type of neural network designed to process image, speech, or audio inputs. According to IBM's explanation of convolutional neural networks, it uses three main layer types: convolutional layers, pooling layers, and fully connected layers. The convolutional layer is the core building block, and the fully connected layer is the final layer that produces the output.
Unlike a fully connected network, a CNN arranges its neurons in three dimensions: width, height, and depth. The Stanford CS231n course notes describe that each layer transforms a 3D volume to another 3D volume. A typical architecture follows the order: input, convolutional, ReLU, pooling, and fully connected layers.
Convolutional layer and feature maps
The convolutional layer applies a set of filters (kernels) to the input. The PyTorch Conv2d documentation defines the layer with parameters such as kernel size, stride, padding, dilation, and groups. The output height and width are computed from the input dimensions, padding, dilation, kernel size, and stride, as specified in that documentation.
Each filter slides across the image and computes dot products with the local region, producing a feature map (also called an activation map). These maps highlight where a particular feature is present. The Dive into Deep Learning chapter on convolutions explains that CNNs exploit translation invariance and locality. This means the same filter can detect a pattern anywhere in the image, and early layers focus on local regions, which greatly reduces the number of parameters compared to fully connected layers.
Pooling layer and its role
Pooling, also called subsampling, reduces the spatial size of the feature maps. The NVIDIA Developer page on CNN notes that pooling provides limited translation and rotation invariance, which improves the detection of objects in unusual positions. It also lowers memory consumption, allowing deeper networks to fit into GPU memory.
The most common pooling operation is max pooling, which takes the maximum value in a region. After pooling, the network applies a nonlinear activation, typically ReLU, to introduce nonlinearity. A larger pooling region condenses more information, but too large a region discards useful detail.
Depth and modern architectures
Large-scale image classification has benefited from deeper CNN architectures. The paper ImageNet Classification with Deep Convolutional Neural Networks describes a network with five convolutional layers, some followed by max pooling, and about 60 million parameters. This network achieved a top-5 error rate of 18.9 percent on the ImageNet test set.
Later work showed that using very small 3x3 filters and increasing depth improves accuracy. The arXiv abstract of the VGG paper found that pushing the network to 16-19 weight layers, with 3x3 convolutions, significantly outperformed prior architectures.
| Layer type | Function | Common parameters | Effect on feature map size |
|---|---|---|---|
| Convolutional | Detect patterns and create feature maps | kernel size, stride, padding, groups | Size changes according to output formula; may keep or reduce size |
| Pooling | Downsample feature maps and add invariance | pooling region size, stride, type | Reduces spatial dimensions |
| Fully connected | Combine features for classification | number of neurons, activation | Reduces 3D volume to class scores (1x1x depth) |
Frequently asked questions
What is a feature map?
A feature map is the output of a convolutional layer after applying a filter to the input. It highlights where the filter's pattern appears in the image.
How does stride affect the output?
Stride is the number of pixels the filter moves each step. A larger stride reduces the spatial size of the feature map.
Why is pooling used?
Pooling reduces the dimensions of feature maps, lowers memory usage, and provides limited translation invariance, helping the network recognize objects regardless of small position shifts.
What are the differences between a convolutional layer and a fully connected layer?
A convolutional layer connects neurons to local regions of the input, sharing weights across the image, while a fully connected layer connects every neuron to all activation units from the previous layer, which leads to many more parameters.