KGP Talkie

2D CNN on CIFAR-10 with TensorFlow 2.0

Train a 2D Convolutional Neural Network on CIFAR-10 using TensorFlow 2.0. Covers Conv2D, MaxPooling, Dropout, model training, and confusion matrix evaluation.

Sep 6, 2020Updated Jul 12, 202618 min readFollow

Topics You Will Master

Convolutional layer fundamentals: filters, strides, and padding
MaxPooling2D and Dropout for spatial down-sampling and regularization
Building multi-block CNN architectures with Keras Sequential API
CIFAR-10 data loading, normalization, and one-hot encoding

In this blog, we will build a multi-block 2D CNN in TensorFlow 2.0 with Conv2D, MaxPooling, and Dropout layers. We train it to classify 10 object categories from the CIFAR-10 dataset. Convolutional Neural Networks (CNNs) use learned filters to pull spatial features from images, which makes them the top choice for object recognition.

Download Data and Model Building

BASH 
!pip install tensorflow
!pip install mlxtend
PYTHON 
import tensorflow as tf
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Flatten, Dense, Conv2D, MaxPool2D, Dropout
print(tf.__version__)
OUTPUT 
2.1.1
PYTHON 
import numpy as np
import matplotlib.pyplot as plt
import matplotlib
from tensorflow.keras.datasets import cifar10

The CIFAR10 dataset has 60,000 color images in 10 classes, with 6,000 images in each class. It is split into 50,000 training images and 10,000 testing images. The classes do not overlap.

PYTHON 
(X_train, y_train), (X_test, y_test) = cifar10.load_data()
OUTPUT 
Downloading data from https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz
170500096/170498071 [==============================] - 50s 0us/step
PYTHON 
classes_name = ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
PYTHON 
X_train.max()
OUTPUT 
255
PYTHON 
X_train = X_train/255
X_test = X_test/255
X_train.shape, X_test.shape
OUTPUT 
((50000, 32, 32, 3), (10000, 32, 32, 3))

Verify the data

Plot the first test image to confirm the data loaded correctly:

PYTHON 
plt.imshow(X_test[0])
PYTHON 
y_test
OUTPUT 
array([[3],
       [8],
       [8],
       ...,
       [5],
       [1],
       [7]], dtype=uint8)

Build CNN Model

The code below defines the convolutional base with a common pattern: a stack of Conv2D, MaxPooling2D, Dropout, Flatten, and Dense layers.

A Conv2D takes tensors of shape (image_height, image_width, color_channels), and ignores the batch size. Here we set it to process inputs of shape (32, 32, 3), the format of CIFAR images.

The Maxpool2D() layer downsamples the input by taking the largest value in each pool_size(2,2) window along the feature axis. The window moves by strides(2) each step. With the "valid" padding option, the output has a smaller shape.

Dropout() randomly sets some hidden-unit outputs to 0 at each training step. The value we pass sets the chance that an output is dropped.

Flatten() is used to convert the data into a 1-dimensional array for inputting it to the next layer.

Dense() is the regular fully connected layer with 128 neurons. The output layer is also a dense layer, with 10 neurons for the 10 classes.

The activation function is softmax. Softmax turns a real vector into a vector of class probabilities. The output values are between 0 and 1 and sum to 1. We often use it in the last layer of a classifier, since we can read the result as a probability spread.

PYTHON 
model = Sequential()
model.add(Conv2D(filters=32, kernel_size=(3, 3), padding='same', activation='relu', input_shape = [32, 32, 3]))

model.add(Conv2D(filters=32, kernel_size=(3, 3), padding='same', activation='relu'))
model.add(MaxPool2D(pool_size=(2,2), strides=2, padding='valid'))
model.add(Dropout(0.5))

model.add(Flatten())
model.add(Dense(units = 128, activation='relu'))
model.add(Dense(units=10, activation='softmax'))
PYTHON 
model.summary()
PYTHON 
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
conv2d (Conv2D)              (None, 32, 32, 32)        896
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 32, 32, 32)        9248
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 16, 16, 32)        0
_________________________________________________________________
dropout (Dropout)            (None, 16, 16, 32)        0
_________________________________________________________________
flatten (Flatten)            (None, 8192)              0
_________________________________________________________________
dense (Dense)                (None, 128)               1048704
_________________________________________________________________
dense_1 (Dense)              (None, 10)                1290
=================================================================
Total params: 1,060,138
Trainable params: 1,060,138
Non-trainable params: 0
_________________________________________________________________

Compile and train the model

We compile the model and fit it to the training data for 10 epochs. An epoch is one pass over the whole data. validation_data is the data used to check the loss and metrics at the end of each epoch, and the model does not train on it. With metrics = ['sparse_categorical_accuracy'], we judge the model by accuracy.

PYTHON 
model.compile(optimizer='adam', loss = 'sparse_categorical_crossentropy', metrics=['sparse_categorical_accuracy'])
PYTHON 
history = model.fit(X_train, y_train, batch_size=10, epochs=10, verbose=1, validation_data=(X_test, y_test))
OUTPUT 
Train on 50000 samples, validate on 10000 samples
Epoch 1/10
50000/50000 [==============================] - 177s 4ms/sample - loss: 1.4127 - sparse_categorical_accuracy: 0.4918 - val_loss: 1.1079 - val_sparse_categorical_accuracy: 0.6095
Epoch 2/10
50000/50000 [==============================] - 159s 3ms/sample - loss: 1.1058 - sparse_categorical_accuracy: 0.6091 - val_loss: 1.0284 - val_sparse_categorical_accuracy: 0.6377
Epoch 3/10
50000/50000 [==============================] - 146s 3ms/sample - loss: 0.9946 - sparse_categorical_accuracy: 0.6477 - val_loss: 0.9682 - val_sparse_categorical_accuracy: 0.6564

The charts below show model accuracy and model loss. The accuracy chart plots training vs validation accuracy. The loss chart plots training vs validation loss.

PYTHON 
# Plot training & validation accuracy values
epoch_range = range(1, 11)
plt.plot(epoch_range, history.history['sparse_categorical_accuracy'])
plt.plot(epoch_range, history.history['val_sparse_categorical_accuracy'])
plt.title('Model accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Val'], loc='upper left')
plt.show()

# Plot training & validation loss values
plt.plot(epoch_range, history.history['loss'])
plt.plot(epoch_range, history.history['val_loss'])
plt.title('Model loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Val'], loc='upper left')
plt.show()

test_loss, test_acc = model.evaluate(X_test,  y_test, verbose=2)
OUTPUT 
10000/10000 - 5s - loss: 0.9383 - sparse_categorical_accuracy: 0.6830
PYTHON 
from mlxtend.plotting import plot_confusion_matrix
from sklearn.metrics import confusion_matrix
PYTHON 
y_pred = model.predict_classes(X_test)
PYTHON 
y_pred
OUTPUT 
array([3, 8, 8, ..., 5, 1, 7])
PYTHON 
y_test
OUTPUT 
array([[3],
       [8],
       [8],
       ...,
       [5],
       [1],
       [7]], dtype=uint8)
PYTHON 
mat = confusion_matrix(y_test, y_pred)
mat
OUTPUT 
array([[737,  27,  22,  17,  14,   4,  12,  14, 106,  47],
       [ 20, 821,   3,  12,   0,   7,   5,   4,  47,  81],
       [ 95,   8, 476,  97,  83, 110,  67,  30,  21,  13],
       [ 34,  14,  42, 520,  52, 203,  58,  43,  21,  13],
       [ 22,   4,  74, 118, 570,  69,  54,  66,  21,   2],
       [ 23,   5,  34, 213,  24, 610,  17,  47,  16,  11],
       [ 10,   8,  34,  80,  42,  40, 760,   8,  13,   5],
       [ 26,   5,  23,  45,  51,  76,   5, 743,  12,  14],
       [ 56,  41,   9,  10,   3,   4,   3,   2, 843,  29],
       [ 43, 116,   5,  18,   6,   4,   5,  21,  32, 750]])
PYTHON 
plot_confusion_matrix(mat,figsize=(9,9), class_names=classes_name, show_normed=True)

Conclusion

In this blog, we built a 2D CNN in TensorFlow 2.0 to classify images from the CIFAR-10 dataset into 10 object categories. After 10 epochs the model reached about 68% test accuracy. The confusion matrix showed that visually similar classes, especially bird, cat, deer, and dog, are the hardest to tell apart, while airplane and ship scored the highest.

Key takeaways:

  • A two-block Conv2D + MaxPool2D architecture with Dropout achieves solid baseline performance on CIFAR-10.
  • Validation accuracy pulling away from training accuracy after epoch 3 is a clear sign of overfitting. Reduce the model size or add stronger regularization.
  • The confusion matrix shows per-class weaknesses far better than a single accuracy number. Always inspect it when classes look similar.

Next steps:

Found this useful? Keep building with me.

New tutorials every week on YouTube: or go deeper with a full structured course.

Subscribe on YouTubeBrowse Udemy CoursesJoin the Newsletter
Related Tutorials

Latest recommendations you might like

Image Classification with Pre-trained VGG-16
Deep Learning

Image Classification with Pre-trained VGG-16

Classify objects using the pre-trained VGG-16 model in Keras. Covers VGG architecture, loading ImageNet weights, image preprocessing, and top-5 predictions.

Sep 20, 2020Read Tutorial
Multi-Label Movie Poster Classification with CNN
Deep Learning

Multi-Label Movie Poster Classification with CNN

Classify movie genres from poster images using a 2D CNN. Covers multi-label classification, ImageDataGenerator, Conv2D with BatchNorm, and sigmoid output.

Sep 7, 2020Read Tutorial
Breast Cancer Detection Using CNN
Deep Learning

Breast Cancer Detection Using CNN

Detect breast cancer using a 1D CNN in TensorFlow 2.0. Covers Conv1D, BatchNormalization, Dropout, Adam optimizer, and binary classification on medical data.

Sep 5, 2020Read Tutorial
Dog vs Cat Classification with CNN
Deep Learning

Dog vs Cat Classification with CNN

Classify dog and cat images using a 2D CNN in TensorFlow 2.0. Covers VGG16 architecture, Dropout, BatchNormalization, ImageDataGenerator, and SGD optimizer.

Sep 4, 2020Read Tutorial

Find this tutorial useful?

Subscribe to our YouTube channels for more practical production walk-throughs.

Watch in EnglishWatch in Hindi

Discussion & Comments