Science News & Breakthroughs Featured May 23, 2026

Solving an Image Classification Problem Using Deep Learning

A step-by-step look at binary image classification with CNNs: data audits, splits, convolutions, cross-entropy, and the evaluation traps and failure modes that decide real-world accuracy.

SC
ScienceTrace Science Desk
•
 7 min read
•
 1,363 words

Suppose we want a system that labels a photograph as "Cat" or "Dog". This is binary image classification, and it is the simplest setting in which to understand the pipeline behind far more consequential systems. The steps are straightforward to list; the instructive part is where each one can quietly go wrong. This walk-through covers the pipeline and the decisions inside it, then turns to evaluation and failure modes, which is where most real projects succeed or fail.

Step 1: Collect and audit the dataset

Supervised learning needs labelled examples. Each image is paired with its correct category:

Image Label
Cat Image Cat
Dog Image Dog

Public datasets used for this kind of work include:

  • CIFAR-10 — 60,000 small (32 × 32 pixel) colour images in 10 classes, including cat and dog, which makes it a standard teaching benchmark but a poor guide to photographs at realistic resolution.
  • ImageNet — over a million training images in the 1,000-class ILSVRC subset (the full database is larger), widely used for pre-training.
  • Kaggle "Dogs vs. Cats" — a 25,000-image training set from a 2013 competition, a convenient size for a first project.

Before training, audit the data. Check for duplicates and near-duplicates (these are the most common cause of inflated results), for mislabelled items, and for confounds, such as all dog photos being taken outdoors and all cat photos indoors. A model will happily learn the confound.

Step 2: Split first, then preprocess

Divide the data into training, validation and test sets before any statistics are computed from it, and keep near-duplicates in the same split. The training set fits the weights; the validation set guides choices such as learning rate and when to stop; the test set is touched once, at the end. Reusing the test set to tune the model turns it into a second validation set and makes the final number optimistic.

Preprocessing makes images consistent:

  • Resize to a fixed size, since standard networks expect fixed-shape input.
  • Normalize pixel values, either to a 0–1 range or by subtracting a per-channel mean and dividing by a standard deviation, computed on the training set only.
  • Convert to tensors, multi-dimensional arrays of shape height × width × channels (or channels first, depending on the framework).
  • Augment the training images only, using random flips, crops and small colour changes, which effectively enlarges the dataset and discourages memorisation.
Example:
Original image size: 1024 × 1024
Resized to: 128 × 128 × 3 (49,152 numbers per image)

Step 3: Build the CNN

A convolutional neural network (CNN) exploits two properties of images: nearby pixels are related, and a useful pattern, such as an edge, can appear anywhere. Instead of connecting every pixel to every unit, a convolutional layer slides a small filter across the image and applies the same weights at every position. Compare the cost. A fully connected layer taking the 49,152 inputs above to just 64 units would need about 3.1 million weights. A 3 × 3 convolution from 3 colour channels to 32 feature maps needs 3 × 3 × 3 × 32 + 32 = 896 parameters, regardless of image size.

Each unit in a layer computes, in simplified form:

y = f(W * x + b)
  • x — the input (an image or the feature maps from the previous layer)
  • W — learned filter weights; * denotes convolution (in a fully connected layer it would be matrix multiplication, Wx)
  • b — bias term
  • f — a non-linearity such as ReLU, without which stacked layers would collapse into one linear map

A typical network alternates convolution and pooling (or strided convolution), which shrinks the spatial size while the number of feature maps grows. Early layers tend to respond to edges and colour blobs, deeper ones to textures and parts. A final layer with a single output unit and a sigmoid function yields a probability that the image is a dog. LeCun and colleagues demonstrated this style for digit recognition in the 1990s; Krizhevsky, Sutskever and Hinton's 2012 network showed it could win the ImageNet challenge, and residual networks (He et al., 2016) made much deeper models trainable. In practice, with a few thousand images, fine-tuning a network pre-trained on ImageNet usually beats training from scratch, because the early features transfer well (Yosinski et al., 2014).

Step 4: Train the model

Training compares predictions with labels, computes a loss, and uses backpropagation to obtain the gradient of that loss with respect to every weight. An optimiser such as SGD with momentum or Adam (Kingma and Ba, 2015) then updates the weights. For binary classification the standard loss is binary cross-entropy, averaged over the N examples in a batch:

Loss = −(1/N) ∑ [ y · log(ŷ) + (1−y) · log(1−ŷ) ]

Here y is the true label (0 or 1) and ŷ the predicted probability. The loss punishes confident wrong answers severely: predicting 0.99 for a cat labelled as a dog costs far more than predicting 0.6. Training runs for several epochs (passes over the data). Watch both training and validation loss. If training loss keeps falling while validation loss rises, the model is overfitting. Remedies include data augmentation, dropout (Srivastava et al., 2014), weight decay, batch normalisation (Ioffe and Szegedy, 2015, mainly for training stability), a smaller network, and early stopping.

Step 5: Test and interpret the result

The trained model is evaluated once on the held-out test set. Accuracy is the obvious number, but it can mislead. With balanced classes it is acceptable; with 95 per cent dogs, a model that always says "Dog" scores 95 per cent. Report a confusion matrix, precision and recall per class, and the area under the ROC curve, which measures how well the scores rank the classes independent of threshold. Also check calibration: when the model says 90 per cent, is it right about 90 per cent of the time? Deep networks are often over-confident.

Illustrative result (not a measured benchmark):
Input image → Dog
Model prediction → Dog (probability 0.97)
A single correct prediction says little; the test-set metrics are what count.

Accuracy figures depend strongly on the dataset and on how much effort goes into the model, so numbers such as "above 95%" should always be accompanied by the dataset, the split and the architecture.

Where classifiers fail in practice

  • Shortcut learning. Networks exploit whatever cue predicts the label in the training data, including background, camera or watermark; Geirhos and colleagues (2020) review many examples. A cat/dog model that learned "grass means dog" will fail on an indoor dog.
  • Texture bias. Standard ImageNet-trained CNNs rely more on texture than on object shape, unlike human observers (Geirhos et al., 2019), so unusual textures can flip predictions.
  • Distribution shift. Performance drops when test images differ from training images in lighting, resolution, camera or breed mix.
  • Adversarial examples. Tiny, deliberately crafted perturbations can change the prediction (Szegedy et al., 2014; Goodfellow et al., 2015), which matters for safety-critical uses.

The same pipeline in other fields

The same structure, with domain-specific data and much stricter validation, underlies applications such as screening of medical images, species identification from camera traps and visual inspection in manufacturing. In each, the lesson from the cat-versus-dog case carries over: results are only as trustworthy as the dataset, the split and the evaluation. In medical imaging, for example, models can learn scanner-specific or hospital-specific cues, so external validation on data from other sites is essential before any claim of clinical usefulness.

Limitations and questions worth testing

  • How does accuracy change when the test set comes from a different source than the training set?
  • Which image regions drive the prediction? Saliency methods such as Grad-CAM can show whether the model looks at the animal or at the background, though they are only approximate explanations.
  • How much does pre-training help as the labelled set shrinks from 20,000 images to 500?
  • Is the model calibrated, and does temperature scaling improve it?

References and further reading

  1. LeCun Y., Bottou L., Bengio Y., Haffner P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998.
  2. Krizhevsky A., Sutskever I., Hinton G. ImageNet Classification with Deep Convolutional Neural Networks. NeurIPS, 2012.
  3. He K., Zhang X., Ren S., Sun J. Deep Residual Learning for Image Recognition. CVPR, 2016.
  4. Kingma D., Ba J. Adam: A Method for Stochastic Optimization. ICLR, 2015.
  5. Srivastava N. et al. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research, 2014.
  6. Yosinski J., Clune J., Bengio Y., Lipson H. How transferable are features in deep neural networks? NeurIPS, 2014.
  7. Geirhos R. et al. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2020.
  8. Geirhos R. et al. ImageNet-trained CNNs are biased towards texture. ICLR, 2019.
  9. Goodfellow I., Bengio Y., Courville A. Deep Learning. MIT Press, 2016.
#deep learning #image classification #CNN #machine learning #AI #computer vision
All Scientific Breakthroughs