Deep Learning Algorithms — The Basic Guide

Deep Learning Algorithms – Deep learning is a very young and limitless field. It is a class of machine learning where theories of the subject aren’t strongly established and views quickly change almost on a daily basis.

Deep Learning can be termed as the best confluence of big data, big models, big computing and big dreams. Deep Learning is an algorithm that has no theoretical limitations of what it can learn; the more data and the more computational (CPU power) time you give, the better it is – Sir Geoffrey Hinton.

“I think people need to understand that deep learning making a lot of things, behind the scenes, much better” – Sir Geoffrey Hinton

What is Deep Learning?

AILabPage defines Deep learning as “Undeniably a mind-blowing synchronisation technique applied on the bases of 3 foundation pillars large data, computing power, skills (enriched algorithms) and experience which practically has no limits“.

Deep Learning is a subfield of the machine learning domain. Deep learning is entirely concerned with algorithms inspired by the structure and function of artificial neural networks which are inspired by the human brain (inspired only pls).

Deep learning is used with too much ease to predict the unpredictable. In our opinion “We all are so busy in creating artificial intelligence by using a combination of non-bio neural networks and natural intelligence rather than exploring what we have in hand.

Deep learning, also called a subset of machine learning is a specialist with an extremely complex skillset to achieve far better results from the same data set. It is purely based on NI (Natural Intelligence) mechanics of the biological neuron system. It has a complex skill set because of the methods it uses for training i.e. learning in deep learning is based on “learning data representations” rather than “task-specific algorithms.” which is the case for other methods

Deep Learning Algorithms

Deep learning leverages autonomous learning mechanisms that depend on simulated neural networks, commonly referred to as artificial neural networks (ANNs), to replicate the intricate cognitive operations of the brain implicated in information processing.

During the process of training, algorithms endeavor to ascertain significant attributes, organize entities, and unveil consequential patterns within the data via the utilization of latent components in the input distribution.

This phenomenon bears similarity to the machine learning technique whereby machines are trained to learn independently, employing various tiers of algorithmic instruments as means to construct and manipulate the models.

It is a special/critical point of discussion for everyone and a puzzling game of all times as well. I am tempted to compare Artificial Neural networks with the human brain. It is fascinating to me to know, how the human brain can decode technologies, numbers, puzzles, handle entertainment, understand science, set body mode into pleasure, aggression, art, etc. How does the brain train itself to name a certain object by just looking at 2-3 images when ANNs need millions of those.

Neural Network Architecture

Neural networks consist of input, output layers hidden layers. Transformation of input into valuable output unit is the main job.

They are excellent examples of mathematical constructs.  Information flows in neural network happens in two ways.

• Feedforward Networks – In these signals only travel in one direction without any loop i.e. towards the output layer. Extensively used in pattern recognition. This network with a single input layer and a single output layer can have zero or multiple hidden layers though. This method has two common designs as below
  • At the time of it’s learning or “being trained”
  • At the time of operating normally or “after being trained”
• Feedback Networks – In this recurrent or interactive networks can use their internal state (memory) to process sequences of inputs.  Signals can travel in both directions with loops in the network. As of now limited to time series/sequential tasks. Typical human brain model.

When to use artificial neural networks as oppose to traditional machine learning algorithms is a complex one to answer.  Neural network architecture and its algorithms may look different to many people but in the end, there is nothing wrong to have them in your tool kit.

Generative Adversarial Networks

A very young family member of Deep Neural Networks Architecture. Introduce by Ian Good-fellow and his team at the University of Montreal in 2014. GANs are a class of unsupervised machine learning algorithms.

So as the name suggests it is called Adversarial Networks because this is made up of two neural networks. Both neural networks are assigned different job role i.e. contesting with each other.

• Neural Network one is called as the Generator because it generates new data instances.
• Other neural net is called the Discriminator, evaluates work for the first neural net for authenticity.

The cycle continues to obtain accuracy or near perfection results. Still confused, it’s ok to read this post on “Generative Adversarial Networks“; you will find more details and understanding.

Recurrent Neural Networks

Recurrent Neural Networks – Call it as a deep tree-like structure. These neural networks are used to understand the context in speech, text or music.

The RNN allows information to loop through the network. Tree-like topology allows branching connections and hierarchical structure. In RNNs data flow is in multiple directions. These networks are employed for highly complex tasks i.e voice recognition, handwriting and language recognition, etc.

RNNs abilities are quite limitless. Don’t get lost between Recursive and Recurrent NNs. ANN’s structure is what enables artificial intelligence, machine learning, and supercomputing to flourish. Neural networks are used for language translation, face recognition, picture captioning, text summarisation and lot more tasks.

Convolutional Neural Networks

Convolutional Neural Networks (CNNs) is an excellent tool and one of the most advanced achievements in deep learning. CNN’s got too much attention and focus from all major business players because of the hype of AI.

The two core concepts of convolutional neural networks are convolution (hence the name) and pooling. It does this job at the backend with many layers transferring information in a sequence from one to another.

The human brain detects any image in fractions of seconds without much effort but computer vision the image is really just an array of numbers. In that array, each cell value represents the brightness of the pixel from black to white for a black and white image. Why do we need CNN’s and not just use feed-forward neural networks? How capsule networks can be used to overcome the shortcoming of CNN’s? etc. I guess if you read this post on “Convolutional Neural Networks“; you will find out the answer.

Recursive Neural Networks

Call it a deep tree-like structure. When the need is to parse a whole sentence we use a recursive neural network.

Tree-like topology allows branching connections and hierarchical structure. Arguments here can be how recursive neural networks are different than recurrent neural networks?

Questions – How recursive neural networks are different than recurrent neural networks?

Answer – Recurrent neural networks are in fact recursive neural networks with a particular structure: that of a linear chain.

RNNs are hierarchical kinds of networks with no time aspect to the input sequence but the input has to be processed hierarchically in a tree fashion.

Backpropagation Algorithm

As mentioned above “Backpropagation” is an algorithm which uses supervised learning methods to compute the gradient descent (delta rule) with respect to weights.

As per wiki – “Backpropagation is a method used in artificial neural networks to calculate a gradient that is needed in the calculation of the weights to be used in the network.”

This algorithm is used for finding minimum value error function in the neural network during the training model stage. The core idea of backpropagation is to find, what impact it would bring to the overall cost of the neural network if we play around with weights.

Weights are used to minimise the error function, so where it minimises that point is considered as the solution to our learning problem. The whole idea of forward/backward propagation and playing with weights is to reduce/minimise or optimise the error value.

After a couple of iteration, the network learns, which side of the number scale it needs to move until error gets minimised. There is a sort of breakpoint where any further update to the weight results in the increase of error and an indication to stop, take it’s as a final weight value.

Boltzmann Machines

Boltzmann Machines – A kind of imaginary recurrent neural network and this normally get interpreted from the probabilistic graphical models.

In a short and concise manner a  neural network which is fully connected and consist of visible and hidden units. It operates in asynchronous mode with stochastic updates for each of its unit. These machines are also called as probability distributions on high dimensional binary vectors.

Thus a probability distributions on high dimensional binary vectors which are analogous to Gaussian Markov Random Fields. Boltzmann machines are fully determined by first and second-order moments.

Conclusion – For any effective machine learning model requirement is only one which is reliable data pipelines. I particularly think that getting to know the types of machine learning algorithms actually helps to see a somewhat clear picture. The answer to the question “What machine learning algorithm should I use?” is always “It depends.” It depends on the size, quality, and nature of the data. Also, what is the objective/motive data torturing? As more we torture data more useful information comes out. It depends on how the math of the algorithm was translated into instructions for the computer you are using. And it depends on how much time you have.  Only recently machine learning got spotlight and attention from the industry. Machine learning use cases like face recognition, image captioning, voice & text processing, and self-driving cars now everyone talks about.

—

Books Referred & Other material referred

• Open Internet reading and research work
• AILabPage (group of self-taught engineers) members hands-on lab work.

Points to Note:

When to use artificial neural networks as oppose to traditional machine learning algorithms is a complex one to answer.  Neural network architecture and its algorithms may look different to many people but in the end, there is nothing wrong to have them in your tool kit. It entirely depends upon the problem in hand to solve. One needs to be patient and experienced enough to have the correct answer. All credits if any remains on the original contributor only. In the next upcoming post will talk about Recursive Neural Networks in detail.

Feedback & Further Question

Do you have any questions about Quantum technologies, Artificial Intelligence and its subdomains like Deep Learning or Machine Learning? etc. Leave a comment or ask your question via email. Will try my best to answer it.

============================ About the Author =======================

Read about Author at : About Me

Thank you all, for spending your time reading this post. Please share your opinion / comments / critics / agreements or disagreement. Remark for more details about posts, subjects and relevance please read the disclaimer.

FacebookPage    ContactMe      Twitter

====================================================================