1. Introduction
In today’s world, the term Artificial intelligence (AI) is being used very widely. This provides the idea about the impact of AI on modern technologies. Every business wants to integrate AI in their business operation to streamline and ease their activities. There is a great advancement with AI, but still, there is a need for a steady set of guides in learning AI.
There are many sources to learn, while this article will provide you with great knowledge on deep learning concepts and, more importantly, provide readers with an understanding of how Deep learning is so important in Machine learning. Many Cloud service providers like E2E provide AI-enabled cloud to provide a good stage for users AI development. “42% of executives believe AI will be of ‘critical importance’ within 2 years.”
This guide is for both technical and non-technical audiences.
2. Deep learning
Machines are used to quickly and efficiently perform certain tasks of operations that are guided by humans. Here machines require human interaction to train or operate the machines. What if machines learn by the training and perform the operations on their own? Any human learns to perform the task by practicing and repeating the tasks, and based on the outcome, he will memorize to perform the tasks. Next time the brain automatically triggers to perform the task more quickly and efficiently.
Deep learning is also constructed in the same pattern. Here machines learn based on the training provided, and the same like the brain, neural networks are constructed to fetch the data. Example: image, sound classification, object detection, image segmentation.
If you are still wondering how important is Deep learning, here are some stats:
- 75% of Netflix users’ film selection is enabled by Netflix’s deep learning algorithms.
- Talking about the value, the global machine learning market in 2017 was valued at $1.58B, and this is projected at $20.83B in 2024.
- In today’s number, at least 2 in 10 companies use AI-enabled software for their business development.
- 83% of Global market leaders say AI & ML is transforming customer engagement.
Even in the era of the pandemic, AI and ML have enabled a lot of innovations.
To get started with learning deep learning as a concept, handwritten digital recognition is considered.
2.1. What is Handwritten Digit Recognition?
Handwritten digit recognition is the potential of computer-aided devices to identify human handwritten digits. It is a difficult task for computers because handwritten digits are not of the same shape or pattern. Everyone has their style of writing the diagram. Handwritten digit recognition is an essential functionality. This uses the image to compare the image of the required pattern.
2.2. Classification of Handwritten Digit Recognition
One of the key functionalities of deep learning and AI is image recognition. This functionality is key for handwritten digit recognition. As a guide, this article will aid in creating mathematical models for identifying handwritten digits. Below examples of handwritten numbers 4 and 2 are shown below.
Here the goal is to create a neural network where the model will recognize the handwritten digits imputed. Like in the given image, the model needs to recognize the image as 4 and 2.
2.2.1. Classification issue with uncertainty
In the above example, sometimes we also fail to recognize the numerical number. Here we need to train the computer for accurate recognition. Here the machine is dealing with classification problems, where given an image, the model needs to classify between 0-9 Digits.
To solve the classification issue, the neural network will return a vector using the 10 positions providing the chances of the digit occurrence.
2.2.2. Data format and manipulation
Moving on to the next phase, the article provides the details in modeling the neural network. Learners can collect the MNIST data to train the model containing 60,000 or more (Greater the training data, more the accuracy) of hand-written digits. The dataset needs to be a black and white set of images and a good resolution of 28×28 pixels.
To accommodate the ingestion of a dataset for the selected neural network, transformation needs to be made from the input (image) in 2D onto the vector image of 1D. The standard format of the matrix 28×28 digits can be represented using the vector (array) of 784 digits (liking is done row by row). This is the standard format to include as an input for the largely connected neural network.
Now, the representation stage, which includes, each label needs to be represented as a vector of 10 positions, this needs to correspond to the position of the digit representing the image containing 1 and the rest containing 0s. The process of changing the label to a vector including as many zeros in the digit with various labels, and labeling 1s in the index to the adjacent label is termed as one-hot encoding. For a better understanding of Digit 4 can be encoded as,
4
⬇
[0.0 0.0 0.0 1.0 0.0 0.0 ]
3. Neural Network
Next, we will look into concepts regarding the neural network. Neural networks are used to train the model.
3.1. Concepts of Neural network.
To showcase the basic operation of a Neural network, let us consider a simple example in which a set of points are showcased in a two-dimensional plane and points are labeled as “circle” or “triangles”:
Now consider a new point “P”, where we need to find what label belongs to it:
A most common way is to mark a straight line separating the two groups that are shown below and the line is used as a classifier, Considering the input data, each input is represented using vectors form (p1, p2) indicating the coordinates in two-dimensional space, returning the functions in ‘0’ or ‘1’ to separate the identity and to know if it needs to classified as “triangle” or “circle”, defined as, And the line is expressed as,
In order to classify the input elements P, in two dimensional, it requires learning the vector weight W which is of the same dimension of the input vector, so vector (w1, w2) and a d bias.
After getting the calculated values, construction of the artificial neuron network can be started for new element P. Here the neuron applies the vector weight W onto the values in the dimension of the input element P, later at the end adding bias values. After this, the result will be passed through a nonlinear function known as the “activation” function, which will produce the result between “0” and “1”. The r is the artificial neuron and it is defined and expressed as,
Considering the function that applies the transformation to variable r which produces ‘0’ or ‘1’. Now considering the sigmoid function which returns the output value between 0 and 1.
Now let’s check the formula, the output always tends to give the value close to 0 and 1. Now if the input s is positive and large, “e” minus s will be zero and the r will take the value 1. Now if the s has a large value and tends to negative. Then the value of r will be 0. Graphical representation of the sigmoid function can be made as shown below,
3.2. Multi-Layer Perceptron
Multilayer perceptron can be referred to as a neural network with one or more input layers which are composed of perceptrons (commonly referred to as a hidden layer) and later the final layer. Or Deep learning can be referred to as a neural network model composed of the multilayer. Below is the visual representation of the scheme,
Image source: https://www.tutorialspoint.com/tensorflow/tensorflow_multi_layer_perceptron_learning.htm
Mostly the MLPs are used during the classification, here we need to classify among the classes (0-9). The outer layer takes up the task of providing the probability using the function called softmax. There can be many activation functions, then the one considered sigmoid, that is softmax activation function. Which is very well suitable when classifying in more than two classes. A detailed explanation of the softmax activation function is provided in the next session.
3.3. Softmax activation function
Now the input set is a set of handwritten images. Given an image the algorithm needs to provide the probability, it is among the 10 possible digits. Now take the example of digit 2 it looks like the digit 2 70%, but the tail part may seem to be 3 30%, this is also true when humans are identifying the digit. So it needs to be modeled in such a way the highest probability is considered, this is a probability distribution function. Here the vector of probability needs to correspond to a digit and their sums need to be in 10 probabilities than the result to be 1.
As discussed earlier this can be achieved by using the output layer with the usage of the softmax activation function. Here each of the neurons in the softmax layer will depend on outputs from the other neurons and their sum needs to be 1.
Working on a softmax activation layer? It’s very straight forward, based on the evidence that any image belongs to a particular class, value is converted into probabilities belonging to possible classes. Here the weighted sum of the evidence output is considered for each of its pixels.
Where the algorithm has created the reference model based on the training set and later this image from the input is compared to the matching probability and this provides the result to input the softmax activation function.
Image source: https://towardsdatascience.com/deep-learning-basics-1d26923cc24a
Once the result is calculated and that each belongs to digits in the 10 classes and the result is 1. The softmax function makes use of the exponential value to normalize.The equation can be written as,
3.4. Neural Network model for identifying handwritten digit
Here a simple neural network can be written based on the sequence of two-layer, which can be represented as,
Image source: https://towardsdatascience.com/handwritten-digit-mnist-pytorch-977b5338e627
Here we can see 784 input features (28×28). The first layer containing 10 neurons using the sigmoid activation function, “distills” are taken to provide the function value between the 10 values. Next coming to the softmax layer of 10 neurons, this means the matrix of 10 probability values is provided.
4. Learning process
This is a vital process of deep learning, where the learning process is carried based on the (weight W and d biases). The weight values are learned and this value is propagated in the network. Propagated value is then shared in the network known as backpropagation to train the value and optimize the given network. Next after optimizing the network the forward propagation takes place which is explained in the next part.
4.1. Training Loop
Primarily we come across the forward propagation, when the neural network is exposed to the training data, they move forward in the network collecting the prediction label for the calculation. Here the data is passed through the network where the transformation is applied, this value is then sent to the next layer from the previous layer. At final the data are crossed among all the layers and once the calculation is complete the final layer is reached using the result of label prediction for the input example.
Next, the loss function is issued to estimate the loss and compare the measure of correctness in relation to the exact result. Here we aim not to get any divergence between the actual and prediction value. Now the model needs to be trained until the weights in the network of neurons have the perfect prediction value. After that the loss value is calculated the information is back propagated for optimizing the network, starting from the final node to the starting node. Once the loss is seen as possible to zero, this network is ready to make the prediction.
4.2. Cross-Entropy Loss Function
The loss function that is used here is the cross-entropy function, which allows the comparison between any given two probability distributions. Cross entropy loss is used to measure the performance of the given classification model, giving the output value between 0-1. At any given time the perfect model should have a log loss of 0.
5. Conclusion
In this article, we have visited the basic and main concepts of the neural network model. This will provide basic and general insights into understanding deep learning and how it is used in detecting hand-written digits. Following this coding needs to be done for working on the model and creating a node.
E2E Network deep learning ready cloud services should be your first choice when choosing AI-enabled cloud. E2E network cloud service comes with ready to use tools that are integrated to handle any volume of machine learning workload. E2E network not only provides a cost-effective cloud solution but also 99 % SLA coverage enabling the perfect stage to engage in designing and implementing uninterrupted machine learning projects.
To know more please signup here: https://bit.ly/2ZiwMTj
