Top 10 Deep Learning Algorithms

Introduction

 

Recently, Deep Learning has gained mass popularity in the world of science computing. When we fire up Alexa or Siri, sometimes we wonder how the machine is able to self-actualize decisions and make right choices ?

 

Well, Deep learning and AI enable the machines to perform these functions making our lives easier and simpler. Deep learning often tends to alleviate the levels of customer experience and the aura of premiumness is unmatched.

 

What is Deep learning?

 

Deep learning is a subset function of AI that imitates the functioning of the human brain and borrows the skill of processing data for use in detecting objects, recognizing speech, translating languages, and making decisions. It is a type of machine learning that works based on the structure and functioning of the human brain.

 

It uses artificial neural networks(ANNs) to perform sophisticated and intricate computations on enormous amounts of data.

 

Deep learning is a subset of artificial intelligence that has networks capable of unsupervised learning from data that is unstructured or unlabeled.

 

Deep learning has evolved hand-in-hand with the digital era, which has brought about a revolution in terms of data extraction in all forms and from every region of the world.

 

This data, renowned  as Big data, is drawn from the sensational sources like social media, internet search engines, e-commerce platforms, and online cinemas, among others.

 

(Also catch: Deep Learning vs Machine Learning)

 

What is a Neural network?

 

A Neural network is a web of artificial neurons known as nodes, which is structured like a human brain. These nodes are stacked next to each other in three layers:

 

• The input layer

• The hidden layer(s)

• The output layer

 

 

What are Top Deep Learning Algorithms?

 

1. Convolutional Neural Network

 

Yann LeCun developed the first CNN in 1988, and named it LeNet. Then it was primarily used for recognizing characters like ZIP codes and digits. 

 

• CNNs now  known as ConvNets, consist of multiple layers structure and are mostly used for image processing and object detection.

• CNN has a convolution layer that has several filters to deal with its intricacy and perform the convolution operation. 

• CNN's also have a Rectified Linear Unit (ReLU) layer to perform operations on elements and present a rectified feature map as an output.

---

CNN structure, Source

---

• The rectified feature map is next fed into a pooling layer and as the name suggests this layer converts the resulting two-dimensional arrays from the pooled feature map into a single, continuous, linear vector by flattening it. 

 

2. Long Short Term Memory Networks

 

LSTMs are a subset of Recurrent Neural Networks (RNN) that are specialised in learning and memorizing long-term information. By default, LSTMs are supposed  to recall past information for long periods of time.

 

• LSTMs have a chain-like structure where four unique layers are stacked. 

• LSTMs are typically used for time-series predictions, speech synthesis, language modeling and translation, music composition, and pharmaceutical development.

---

LSTM structure, Source

---

• They are programmed to forget irrelevant parts of the data and selectively update the cell-state values.

 

3. Recurrent Neural Networks

 

As the Wikipedia page defines a recurrent neural network (RNN), it is a class of artificial neural networks where connections between nodes form a directed graph along a temporal sequence. This allows it to exhibit temporal dynamic behavior.

 

Due to this dynamic behaviour, the output from LSTMs is allowed to be fed as input here.

 

• The output from the LSTM becomes an input to the current phase allowing to  memorize previous inputs due to its efficient internal memory. 

• RNNs are mostly used for image captioning, time-series analysis, natural-language processing, handwriting recognition, and machine translation.

---

Working of RNNs,source

---

• RNNs can process inputs of varied lengths. The more the computation, the more will be the possibility of information to be gathered and in addition the model size does not increase with the input size.

 

4. Generative Adversarial Networks

 

GANs are generative deep learning algorithms that are responsible for producing new data instances that identify with the training data provided. 

 

• GANs have two main components: a generator, which is used to generate fake data, and a discriminator, which learns from that false information.  

• GANs expertise are used to generate realistic images and cartoon characters, create photographs of human faces, and render 3D objects. 

• You might have noticed GANs logos on video games as developers use GANs to upgrade low-resolution, 2D textures in vintage video games by recreating them in surreal 4K or higher resolutions via image training. 

• They are also used to improve astronomical images and simulate gravitational lensing for dark-matter research.

---

Working of GANs, source

---

• During the training period, the Discriminator learns to distinguish between real and fake data and is able to rectify whenever the Generator produces fake data.

 

5. Radial basis function networks

 

RBFNs are an example of artificial neural networks, mainly used for function approximation problems. 

 

• Radial basis function networks are considered better from other neural networks because of their universal approximation method and faster learning speed.

• An RBF network is a special type of feed forward neural network. It consists of three different layers, namely the input layer, the hidden layer and the output layer.

• An RBF network with 10 nodes in its hidden layer is chosen. The training of the RBF model is terminated once the calculated error boils down to ideal values (i.e. 0.01) or number of training iterations (i.e. 500) already was completed.

---

 Working of RBFNs,Source

---

• RBFNs tend to perform classification by measuring the input's congruency to examples from the training set. The function finds the total sum of the inputs, and the output layer receives one node per category or class of data.

• The neurons in the hidden layer work on the principles of Gaussian transfer functions, which produces outputs that are inversely proportional to their distance from the neuron's center.The network's output is an inter-webbed combination of the input’s radial-basis functions and the neuron’s parameters.

 

6. Multilayer Perceptrons

 

MLPs belong to the family of feedforward neural networks with multiple layers of perceptrons that have different functions. 

 

• MLPs consist of an input layer and an output layer that are fully connected with the hidden layers in between. 

• They have the same number of input and output layers but may have multiple hidden layers, which act as the true computation engine of MLPs. 

• They are used to build speech-recognition, financial prediction, and carry out data compression.

• The data is fed to the input layer of the network. Then the layers of neurons form a pattern which enables the signal to pass in one direction.

• MLPs compute the  input with the entities that exist between the input layer and the hidden layers.

---

Working of MLPs,source

---

• Activation functions like ReLUs, sigmoid functions, and Tanhs allow MLPs to determine which nodes to use. 

• MLPs assist the model to understand the correlation and learn the dependencies between the independent and the target variables from a particular training data set.

 

7. Self Organising Maps

 

Professor Teuvo Kohonin invented SOMs or Kohenin’s map, which enable data visualization to reduce the dimensions of data by creating a spatially organised representation.

 

It also helps us to understand the correlation between sets of data. Data visualization attempts to solve the problem that the human mind cannot easily visualize i.e. high-dimensional data. 

 

• SOMs are created to help users access and understand this high-dimensional information.

• SOMs don't have activation functions in neurons, so they initialize weights for each node and choose a vector at random from the training data.

• SOMs examine every node to find which weights are the most likely input vector and the most suitable node is called the Best Matching Unit(BMU).

---

Working of SOMs

---

• SOMs discover the crowd around BMU’s neighborhood, which tends to get lower over time. The closer a node is to a BMU, the more its weight changes and the winning weight is awarded to the sample vector.

 

8. Deep Beliefs Networks

 

DBNs are generative graphical models or a class of deep neural networks that consist of multiple layers of stochastic, latent variables.The latent variables have binary values and are often called hidden units which are connected between the layers        but not within a single layer.

 

• DBNs are an arrangement of Boltzmann Machines with connections between the layers, and in which each RBM layer communicates with both the previous and subsequent layers. 

• DBNs are used for image-recognition, drug discovery, video-recognition, and motion-capture data. 

• Greedy(to choose the most optimal option) learning algorithms train DBNs. The greedy learning algorithm uses an intensive approach of layer-by-layer learning of the top-down, generative weights.

• DBNs run the steps of Gibbs sampling for analysing on the top two hidden layers.

---

Working of DBNs

---

• DBNs draw samples from the visible units using a single pass of ancestral sampling throughout the model. 

• DBNs learn that the values of the latent variables in every layer can be concluded by a single, bottom-up pass.

 

9. Restricted Boltzmann Machines

 

Developed by Geoffrey Hinton, RBMs are stochastic neural networks that possess the capability to learn from a probability distribution over the data ingested.

 

• RBMs is the founder of many applications in the fields of dimensionality reduction, classification and can be trained over supervised or unsupervised data.

• This neural network has applications in regression, collaborative filtering, feature learning, and even many body quantum mechanics. 

• RBMs are considered to be the building blocks of DBNs.

• RBMs consist of two units: Visible units and Hidden units. Each visible unit is symmetrically connected to all hidden units. RBMs consist of a bias unit that is connected to all the visible units and the hidden units but lack output nodes.

---

Working of RBFs

---

• RBMs accept the input and encode it via numbers in the forward pass.RBMs combine each input with the individual's own weight and one overall bias unit.

 

10. Autoencoders

 

Autoencoder is an unsupervised artificial neural network that learns and understands how to efficiently compress and encode data. 

 

Then learns how to reconstruct the data back from the encoded compression to a representation that is as close to the original input provided at first.

---

Autoencoders Configuration, Source

---

Autoencoders are supposed to first encode the image, then reduce the size of the input into a smaller entity. Finally, the autoencoder decodes the image in order to generate the reconstructed image.

 

 

Conclusion

 

All the Deep learning algorithms show us that why are they preferred over other techniques. All the algorithms compel us to use deep learning as they have become the norm of the world lately and also serve to our comfort with time, effort and ease of use. 

 

Deep learning has made the working of computers to actually become smart and make them work according to our needs. 

 

With the ever growing data, it can be concluded that these algorithms would only become more efficient with time and would truly be able to replicate the juggernauts of a human brain.