Evolution of Artificial Neural Network(ANN)

Published

May 7, 2020

Artificial Intelligence

Artificial intelligence (AI) is a branch of computer science capable of performing tasks that typically require human intelligence.

What is Deep Learning?

deep learning is a more approachable name for an artificial neural network. The “deep” in deep learning refers to the depth of the network. An artificial neural network can be very shallow.

  • Machine learning is the science of getting computers to act without being explicitly programmed.

ANN: Artificial Neural Networks

  • ANNs are inspired by biological neurons found in cerebral cortex of our brain.

  • A neuron or nerve cell is an electrically excitable cell that communicates with other cells via specialized connections called synapses.
  • ANNs are core of deep learning. Hence one of the most important topic to understand.

Perceptron

A Perceptron is one of the simplese ANN architectures, invented in 1957 by Frank Rosenblatt.

A perceptron works similar to a biological neuron. A biological neuron receives electrical signals from its dendriles, modulates the electrical signals in various amounts, and then fires an output signal through its synapses only when the total strength of the input signals exceeds a certain threshold. The output is then fed to another neuron, and so forth.

To model the biological phenomenon, the artificial neuron performs two consecutive functions: it calculates the weighted sum of the inputs to represent the totgal strength of the input signals, and it applies a step function to the result to determine whether to fire the ourput 1 if the signal exceeds a certain threshold of 0 if the signal doesn’t exceed the threshold.

How Peceptron Works?

The perceptron’s learning logic goes like this:

  1. The neuron calculate the weighted sum and applies the activation function to make a prediction y^. This is called the feedforward process:

\[y\hat{} = activation\left(\sum\nolimits x_i \cdot w_i + b\right)\]

  1. It compares the output prediction with the correct label to calculate the error:

\[error = y-y\hat{}\]

  1. It then update the weight. If the prediction is too high, it adjust the weight to make a lower prediction the next time, and vice versa.

  2. Repeat!

Here’s how a since perceptron works to classify two classes :

Drawback of perceptron. Sometimes its not possible to get desired result with only perceptron. In the below example you can see the model in not able to draw a line to classify the data(linearly inseperable data)

Indroduction of ANN If you increase the number of neuron then you cann see model works pretty well. The stack of more than one neuron is called Multi Layer Perceptron or ANN.

ANN using Keras

Code 
# TensorFlow and tf.keras
import tensorflow as tf

# Helper libraries
import numpy as np
from tensorflow.keras import initializers
from tensorflow.python.keras import activations

print(tf.__version__)

# downloading fashion_mnist data
fashion_mnist = tf.keras.datasets.fashion_mnist

(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()

class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',
               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']

train_images = train_images / 255.0

test_images = test_images / 255.0 

activation = tf.keras.activations.relu

model = tf.keras.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(128, activation=activation),
tf.keras.layers.Dense(10)
])

model.compile(optimizer='adam',loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),metrics=['accuracy'])

# model summary
model.summary()
c:\users\sonu.ramkumar.jha\desktop\experiments\env\lib\site-packages\numpy\_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs:
c:\users\sonu.ramkumar.jha\desktop\experiments\env\lib\site-packages\numpy\.libs\libopenblas.GK7GX5KEQ4F6UYO3P26ULGBQYHGQO7J4.gfortran-win_amd64.dll
c:\users\sonu.ramkumar.jha\desktop\experiments\env\lib\site-packages\numpy\.libs\libopenblas.WCDJNK7YVMPZQ2ME2ZZHJJRJ3JIKNDB7.gfortran-win_amd64.dll
  warnings.warn("loaded more than 1 DLL from .libs:"
2.5.0
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
flatten (Flatten)            (None, 784)               0         
_________________________________________________________________
dense (Dense)                (None, 128)               100480    
_________________________________________________________________
dense_1 (Dense)              (None, 10)                1290      
=================================================================
Total params: 101,770
Trainable params: 101,770
Non-trainable params: 0
_________________________________________________________________
Code 
model.fit(train_images, train_labels, epochs=10)

test_loss, test_acc = model.evaluate(test_images,  test_labels, verbose=2)

print('test_loss', test_loss)
print('test_accuracy', test_acc)
Epoch 1/10
1875/1875 [==============================] - 4s 2ms/step - loss: 0.5000 - accuracy: 0.8255
Epoch 2/10
1875/1875 [==============================] - 3s 1ms/step - loss: 0.3751 - accuracy: 0.8645
Epoch 3/10
1875/1875 [==============================] - 3s 2ms/step - loss: 0.3388 - accuracy: 0.8773
Epoch 4/10
1875/1875 [==============================] - 3s 2ms/step - loss: 0.3142 - accuracy: 0.8843
Epoch 5/10
1875/1875 [==============================] - 3s 2ms/step - loss: 0.2957 - accuracy: 0.8922
Epoch 6/10
1875/1875 [==============================] - 3s 2ms/step - loss: 0.2797 - accuracy: 0.8971
Epoch 7/10
1875/1875 [==============================] - 3s 2ms/step - loss: 0.2685 - accuracy: 0.8991
Epoch 8/10
1875/1875 [==============================] - 3s 1ms/step - loss: 0.2577 - accuracy: 0.9028
Epoch 9/10
1875/1875 [==============================] - 3s 1ms/step - loss: 0.2476 - accuracy: 0.9076
Epoch 10/10
1875/1875 [==============================] - 3s 1ms/step - loss: 0.2388 - accuracy: 0.9105
313/313 - 0s - loss: 0.3403 - accuracy: 0.8798
test_loss 0.34025073051452637
test_accuracy 0.879800021648407