Neural Networks

Deep Learning – Introduction to Artificial Neural Networks

Artificial Neural Networks – As the name suggest “Neural Network”, they are inspired by brain system. They were originally designed to about biological neurons (also called as perceptrons).


Artificial Neural Networks (ANN) – Some background

As per Dr. Robert Hecht-Nielsen, the inventor of one of the first neuron computer.  ANN is “A computing system made up of a number of simple, highly connected processing elements, which process information by their dynamic state response to external inputs.”

AILabPage defines – Artificial neural networks (ANNs) as “Biologically inspired computer codes for simulating human brain working & process model”. Its way different the computer program though.

ANN’s learns, get trained and adjust automatically like we human do. Though ANN’s are inspired by human brain but for a fact they run on a far simpler plane. The structure of neurons are now used for machine learning thus called as artificial learning.

This development has helped various problems to come to an end especially where layering is needed for refinement and granular details are needed.


The neural networks are now  part of research subject like Neuro-informatics (a research field concerned with the organisation of neuroscience data by the application of computational models and analytical tools).

It requires extensive training before implementing in real-time problem solving environment. they can solve problems.


Artificial Neural Networks – Types and Kinds

There are several kinds of Neural Networks in deep learning. Neural networks consist of input and output layers and at-least one hidden layer.

Neural network based on radial basis function with can be used for strategic reasons. There are several other models of neural network including what we have mentioned above. For an introduction to neural network and their working model continue reading this post. You will get a sense of how they work and used for real mathematical problems.

Artificial Neural Networks - Everything You Need To Know

AILabPage’s Deep Learning Series

In short ANN’s are designed to simulate computer’s working process the way the human brain processes information. ANN’s got popular just in recent times as in past we neither had the computing power nor the required amount of data to train them.


Artificial Neural Networks Components

Neural Networks work as a visual guide for data evolution strategies which are highly scalable alternative for deep reinforcement learning. From the architectural point of view artificial neural networks has three components.

  1. Model Topology (Connections) – To describes the layers of neurons and structure of the connections between them.
  2. Activation Function (Transfer Function) – Function to be used by the artificial neurons.
  3. Learning Algorithm  –  To find the ideal values of the weights.

Each “neuron” is a relatively simple element e.g. summing its inputs and applying a threshold to the result, to decide the output of that “neuron”.


Neural Network Work Flow – Layers of Learning

The underlying foundation of neural networks are layers and layers of connections. The entire neural network model is based on layered architecture. Each layer has its own responsibility.

Layer take input, extract feature and feed into next layer i.e. each layer work as an input layer to another layer. This is to receive information and last layer job is to throw output of the required information. Hidden layers or core layers process all the information in between.

Artificial Neural Networks - Everything You Need To Know

AILabPage’s Deep Learning series Artificial Neural Networks

  • Assign random weight to all the links to start the algorithm.
  • Find links the activation rate of all hidden nodes by using the input and links.
  • Find activation rate of output nodes with activation rate of hidden nodes and link to output.
  • Errors are discovered at the output node and to recalibrate all the links between hidden & output nodes.
  • Using the weights and error at output; cascade down errors to hidden & output nodes.
  • Recalibrate & repeat the process of weights between hidden and input nodes till the convergence criterion are met.
  • Finally the output value of the predicted value or the sum of the three output values of each neuron. This is the output.

Patterns of information are fed into the network via the input units, which trigger the layers of hidden units, and these in turn arrive at the output units.

Deep Learning’s most common model is back propagation. This has become the foundation for most of the others, is the 3-layer fully connected.


Deep Learning or Machine Learning

Making decision or choosing between ML or DL depends mainly on the problem & data at hand. Some high level differences as below.

Should you choose to read in details then I suggest you read “Demystifying the difference between Machine Learning and Deep Learning“.

  • ANN’s – In ANN’s learning is based on multiple layers of features or representation in each layer with the layers forming a hierarchy of low-level to high-level features. It may sound far-fetched, but remember that humans are neurologically hard-wired. In ANN’s Focus is really on end-to-end automatic learning based on raw features.

ANN Sample

  • Machine learning – In ML focus is on feature engineering. Traditional machine learning create / train test splits of the data where ever possible via cross-validation.

In a scenario where deep understanding of medical symptoms is required for some disease detection, a high performance and accuracy is very crucial. So in this case deep learning would be a better choice compare to traditional machine learning.


Convolutional Neural Networks

Convolutional Neural Networks is an excellent and one of the most advanced achievement in deep learning.

Because of  CNN’s deep learning got hyped and so much of attention & focus from all players in business. The two core concepts in this are convolution and pooling.

Why do we need CNN’s and not just use use feed-forward neural networks; I guess if you read this post on “Everything to …. Convolutional Neural Networks“; you will find out the answer.


Recursive Neural Networks

Recursive Neural Networks – Call it as deep tree like structure. When need is to parse a whole sentence we use recursive neural network. Tree like topology allow branching connections and  hierarchical structure.

Questions – Arguments here can be how recursive neural network are different then recurrent neural networks?

Answer – To respond in one line we can say recurrent neural networks are in fact recursive neural networks with a particular structure: that of a linear chain.


Books Referred & Other material referred

  • Open Internet & AILabPage (group of self-taught engineers) members hands on lab work.

Points to Note:

When to use artificial neural networks as oppose to traditional machine learning algorithms is complex one to answer.  It entirely depends upon on the problem in hand to solve. One needs to be patient and experienced enough to have correct answer.

All credits if any remains on the original contributor only. In the next upcoming post will talk about Recurrent Neural Networks in detail.


Feedback & Further Question

Do you have any questions about Deep Learning or Machine Learning? Leave a comment or ask your question via email . Will try my best to answer it.


SECaaS - Security as a Service Is the Next Big ThingConclusion –  We have seen in post above that ANN’s don’t create or invent any new information or facts. ANN help us make sense of what’s already in front of us hidden in our data.

Deep Learning in short is going much beyond machine learning and its algorithms that are either supervised or unsupervised. In DL it uses many layers of nonlinear processing units for feature extraction and transformation.

ANN’s structure is what enables artificial intelligence, machine learning and supercomputing to flourish. Neural networks are powers language translation, face recognition, picture captioning, text summarisation and lot more.


============================ About the Author =======================

Read about Author at : About Me

Thank you all, for spending your time reading this post. Please share your opinion / comments / critics / agreements or disagreement. Remark for more details about posts, subjects and relevance please read the disclaimer.

FacebookPage    ContactMe      Twitter


Facebook Comments

3 replies »

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.