Building Your First ANN with TensorFlow 2.0

Deep learning with Tensorflow

In this blog, we will build our first Artificial Neural Network (ANN) with TensorFlow 2.0 and Keras. An ANN learns from data by passing signals through layers of weighted neurons. It then tunes the weights with backpropagation to lower the loss. Here we build a simple yes-or-no classifier and cover activation functions, optimizers, and how to test the model.

Installing libraries

PYTHON

# pip install tensorflow==2.0.0-rc0
# pip install tensorflow-gpu==2.0.0-rc0

PYTHON

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Flatten, Dense

PYTHON

print(tf.__version__)

OUTPUT

2.2.0

Importing necessary libraries

PYTHON

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split

PYTHON

dataset = pd.read_csv('Customer_Churn_Modelling.csv')

PYTHON

dataset.head()

OUTPUT

	RowNumber	CustomerId	Surname	CreditScore	Geography	Gender	Age	Tenure	Balance	NumOfProducts	HasCrCard	IsActiveMember	EstimatedSalary	Exited
0	1	15634602	Hargrave	619	France	Female	42	2	0.00	1	1	1	101348.88	1
1	2	15647311	Hill	608	Spain	Female	41	1	83807.86	1	0	1	112542.58	0
2	3	15619304	Onio	502	France	Female	42	8	159660.80	3	1	0	113931.57	1
3	4	15701354	Boni	699	France	Female	39	1	0.00	2	0	0	93826.63	0
4	5	15737888	Mitchell	850	Spain	Female	43	2	125510.82	1	1	1	79084.10	0

PYTHON

X = dataset.drop(labels=['CustomerId', 'Surname', 'RowNumber', 'Exited'], axis = 1)
y = dataset['Exited']

PYTHON

X.head()

OUTPUT

	CreditScore	Geography	Gender	Age	Tenure	Balance	NumOfProducts	HasCrCard	IsActiveMember	EstimatedSalary
0	619	France	Female	42	2	0.00	1	1	1	101348.88
1	608	Spain	Female	41	1	83807.86	1	0	1	112542.58
2	502	France	Female	42	8	159660.80	3	1	0	113931.57
3	699	France	Female	39	1	0.00	2	0	0	93826.63
4	850	Spain	Female	43	2	125510.82	1	1	1	79084.10

PYTHON

y.head()

OUTPUT

0    1
1    0
2    1
3    0
4    0
Name: Exited, dtype: int64

Using label encoder we are converting categorical features to numerical features

PYTHON

from sklearn.preprocessing import LabelEncoder

PYTHON

label1 = LabelEncoder()
X['Geography'] = label1.fit_transform(X['Geography'])
label = LabelEncoder()
X['Gender'] = label.fit_transform(X['Gender'])
X.head()

OUTPUT

	CreditScore	Geography	Age	Tenure	Balance	NumOfProducts	HasCrCard	IsActiveMember	EstimatedSalary
0	619	0	42	2	0.00	1	1	1	101348.88
1	608	2	41	1	83807.86	1	0	1	112542.58
2	502	0	42	8	159660.80	3	1	0	113931.57
3	699	0	39	1	0.00	2	0	0	93826.63
4	850	2	43	2	125510.82	1	1	1	79084.10

PYTHON

X = pd.get_dummies(X, drop_first=True, columns=['Geography'])
X.head()

OUTPUT

	CreditScore	Age	Tenure	Balance	NumOfProducts	HasCrCard	IsActiveMember	EstimatedSalary	Geography_2
0	619	42	2	0.00	1	1	1	101348.88	0
1	608	41	1	83807.86	1	0	1	112542.58	1
2	502	42	8	159660.80	3	1	0	113931.57	0
3	699	39	1	0.00	2	0	0	93826.63	0
4	850	43	2	125510.82	1	1	1	79084.10	1

Here using standardscaler we are scaling our data, we are scaling such that the mean is 0 and variance is 1 for data

PYTHON

from sklearn.preprocessing import StandardScaler

PYTHON

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0, stratify = y)

PYTHON

scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

PYTHON

X_train

OUTPUT

array([[-1.24021723, -1.09665089,  0.77986083, ...,  1.64099027,
        -0.57812007, -0.57504086],
       [ 0.75974873,  0.91186722, -0.27382717, ..., -1.55587522,
         1.72974448, -0.57504086],
       [-1.72725557, -1.09665089, -0.9443559 , ...,  1.1038111 ,
        -0.57812007, -0.57504086],
       ...,
       [-0.51484098,  0.91186722,  0.87565065, ..., -1.01507508,
         1.72974448, -0.57504086],
       [ 0.73902369, -1.09665089, -0.36961699, ..., -1.47887193,
        -0.57812007, -0.57504086],
       [ 0.95663657,  0.91186722, -1.32751517, ...,  0.50945854,
        -0.57812007,  1.73900686]])

Build ANN

Here we are building the ANN model.
First we add an input layer with shape equal to the number of input features, which is 11 in this case.
There is one hidden layer with 128 units.
The output layer has shape 1 since this is a binary classification problem.

PYTHON

model = Sequential()
model.add(Dense(X.shape[1], activation='relu', input_dim = X.shape[1]))
model.add(Dense(128, activation='relu'))
model.add(Dense(1, activation = 'sigmoid'))

PYTHON

X.shape[1]

OUTPUT

Here we are compiling our model. we have selected Adam optimizer. loss is binary crossentropy and metric is accuracy

PYTHON

model.compile(optimizer='adam', loss = 'binary_crossentropy', metrics=['accuracy'])

Here we are fitting model on training dataset . we have given bacth size of 10 and eopchs are 10

PYTHON

model.fit(X_train, y_train.to_numpy(), batch_size = 10, epochs = 10, verbose = 1)

OUTPUT

Epoch 1/10
800/800 [==============================] - 1s 2ms/step - loss: 0.4516 - accuracy: 0.8116
Epoch 2/10
800/800 [==============================] - 1s 1ms/step - loss: 0.3948 - accuracy: 0.8372
Epoch 3/10
800/800 [==============================] - 1s 1ms/step - loss: 0.3597 - accuracy: 0.8543
Epoch 4/10
800/800 [==============================] - 1s 2ms/step - loss: 0.3475 - accuracy: 0.8576
Epoch 5/10
800/800 [==============================] - 1s 1ms/step - loss: 0.3426 - accuracy: 0.8611
Epoch 6/10
800/800 [==============================] - 1s 1ms/step - loss: 0.3389 - accuracy: 0.8619
Epoch 7/10
800/800 [==============================] - 1s 1ms/step - loss: 0.3366 - accuracy: 0.8625
Epoch 8/10
800/800 [==============================] - 1s 1ms/step - loss: 0.3350 - accuracy: 0.8629
Epoch 9/10
800/800 [==============================] - 1s 2ms/step - loss: 0.3333 - accuracy: 0.8635
Epoch 10/10
800/800 [==============================] - 1s 1ms/step - loss: 0.3311 - accuracy: 0.8634

Using model.predict we predict output values for our input data.

PYTHON

y_pred = model.predict_classes(X_test)

PYTHON

y_pred

OUTPUT

array([[0],
       [0],
       [0],
       ...,
       [0],
       [1],
       [0]])

PYTHON

y_test

OUTPUT

1344    1
8167    0
4747    0
5004    1
3124    1
       ..
9107    0
8249    0
8337    0
6279    1
412     0
Name: Exited, Length: 2000, dtype: int64

PYTHON

model.evaluate(X_test, y_test.to_numpy())

OUTPUT

63/63 [==============================] - 0s 2ms/step - loss: 0.3489 - accuracy: 0.8520

[0.34891313314437866, 0.8519999980926514]

PYTHON

from sklearn.metrics import confusion_matrix, accuracy_score

Confusion matrix

PYTHON

confusion_matrix(y_test, y_pred)

OUTPUT

array([[1546,   47],
       [ 249,  158]], dtype=int64)

PYTHON

accuracy_score(y_test, y_pred)

OUTPUT

0.852

Conclusion

In this blog, we built a three-layer ANN in TensorFlow 2.0 to predict customer churn from the Bank Customer Churn dataset. We encoded the category columns with LabelEncoder and get_dummies, and scaled the data with StandardScaler. The model trained for 10 epochs and reached 85.2% test accuracy. We checked this with both model.evaluate() and the confusion_matrix.

Key takeaways:

Category columns like geography and gender must be turned into numbers before they go into a neural network. LabelEncoder plus one-hot encoding via get_dummies is the standard way to do this.
A simple three-layer ANN (input, 128-unit hidden, sigmoid output) is a strong baseline for yes-or-no tasks before we move to deeper or convolutional models.
The confusion matrix shows the model gets most non-churners right but misses 249 churners. Handling class imbalance would raise recall on the smaller class.

Next steps:

Extend to a 1D CNN for the same dataset type in Credit Card Fraud Detection using CNN to compare performance.
Explore deeper ANN designs and regularization in TensorFlow 2.0 Getting Started.
Add Dropout layers between the hidden and output layers to reduce overfitting and improve generalization.