Artificial Neural Networks – As the name suggest “Neural Network”, they are inspired by brain system. They were originally designed to about biological neurons (also called as perceptrons).
Neural networks consist of input and output layers and at-least one hidden layer.
In short ANN’s are designed to simulate computer’s working process the way the human brain processes information.
ANN’s got popular just in recent times as in past we neither had the computing power nor the required amount of data to train them.
Artificial Neural Networks (ANN) – Some background
As per Dr. Robert Hecht-Nielsen, the inventor of one of the first neuron computer. ANN is “A computing system made up of a number of simple, highly connected processing elements, which process information by their dynamic state response to external inputs.”
AILabPage defines – Artificial neural networks (ANNs) as “Biologically inspired computer codes for simulating human brain working & process model”. Its way different the computer program though.
ANN’s learns, get trained and adjust automatically like we human do. Though ANN’s are inspired by human brain but for a fact they run on a far simpler plane.
The structure of neurons are now used for machine learning thus called as artificial learning.
This development has helped various problems to come to an end especially where layering is needed for refinement and granular details are needed.
The neural networks are now part of research subject like Neuro-informatics (a research field concerned with the organisation of neuroscience data by the application of computational models and analytical tools).
It requires extensive training before implementing in real-time problem solving environment. they can solve problems.
Neural Network Architecture and its Components
Neural networks consist of input, output layers hidden layers. Transformation of input into valuable output unit is the main job. They are excellent example of mathematical constructs. Information flows in neural network happens in two ways.
- Feedforward Networks – In this signals only travel in one direction without any loop i.e. towards the output layer. Extensively used in pattern recognition. This method has two common designs as below
- At the time of it’s learning or “being trained”
- At the time of operating normally or “after being trained”
- Feedback Networks – In this recurrent or interactive networks can use their internal state (memory) to process sequences of inputs. Signals can travel in both directions with loops in the network. As of now limited to time series/sequential tasks. Typical human brain model.
Artificial Neural Networks Components
Neural Networks work as a visual guide for data evolution strategies which are highly scalable alternative for deep reinforcement learning. From the architectural point of view artificial neural networks has three components.
- Model Topology (Connections) – To describes the layers of neurons and structure of the connections between them.
- Activation Function (Transfer Function) – Function to be used by the artificial neurons.
- Learning Algorithm – To find the ideal values of the weights.
Each “neuron” is a relatively simple element e.g. summing its inputs and applying a threshold to the result, to decide the output of that “neuron”.
Neural Network Work Flow – Layers of Learning
The underlying foundation of neural networks are layers and layers of connections. The entire neural network model is based on layered architecture. Each layer has its own responsibility.
Layer take input, extract feature and feed into next layer i.e. each layer work as an input layer to another layer. This is to receive information and last layer job is to throw output of the required information. Hidden layers or core layers process all the information in between.
- Assign random weight to all the links to start the algorithm.
- Find links the activation rate of all hidden nodes by using the input and links.
- Find activation rate of output nodes with activation rate of hidden nodes and link to output.
- Errors are discovered at the output node and to recalibrate all the links between hidden & output nodes.
- Using the weights and error at output; cascade down errors to hidden & output nodes.
- Recalibrate & repeat the process of weights between hidden and input nodes till the convergence criterion are met.
- Finally the output value of the predicted value or the sum of the three output values of each neuron. This is the output.
Patterns of information are fed into the network via the input units, which trigger the layers of hidden units, and these in turn arrive at the output units.
Deep Learning’s most common model is back propagation. This has become the foundation for most of the others, is the 3-layer fully connected.
Deep Learning or Machine Learning
Making decision or choosing between ML or DL depends mainly on the problem & data at hand. Some high level differences as below.
Should you choose to read in details then I suggest you read “Demystifying the difference between Machine Learning and Deep Learning“.
- ANN’s – In ANN’s learning is based on multiple layers of features or representation in each layer with the layers forming a hierarchy of low-level to high-level features. It may sound far-fetched, but remember that humans are neurologically hard-wired. In ANN’s Focus is really on end-to-end automatic learning based on raw features.
- Machine learning – In ML focus is on feature engineering. Traditional machine learning create / train test splits of the data where ever possible via cross-validation.
In a scenario where deep understanding of medical symptoms is required for some disease detection, a high performance and accuracy is very crucial. So in this case deep learning would be a better choice compare to traditional machine learning.
Convolutional Neural Networks is an excellent and one of the most advanced achievement in deep learning.
Because of CNN’s deep learning got hyped and so much of attention & focus from all players in business. The two core concepts in this are convolution and pooling.
Why do we need CNN’s and not just use use feed-forward neural networks; I guess if you read this post on “Everything to …. Convolutional Neural Networks“; you will find out the answer.
Recursive Neural Networks
Recursive Neural Networks – Call it as deep tree like structure. When need is to parse a whole sentence we use recursive neural network. Tree like topology allow branching connections and hierarchical structure.
Questions – Arguments here can be how recursive neural network are different then recurrent neural networks?
Answer – To respond in one line we can say recurrent neural networks are in fact recursive neural networks with a particular structure: that of a linear chain.
Books Referred & Other material referred
- Open Internet & AILabPage (group of self-taught engineers) members hands on lab work.
Points to Note:
When to use artificial neural networks as oppose to traditional machine learning algorithms is complex one to answer. It entirely depends upon on the problem in hand to solve. One needs to be patient and experienced enough to have correct answer.
All credits if any remains on the original contributor only. In the next upcoming post will talk about Recurrent Neural Networks in detail.
Feedback & Further Question
Do you have any questions about Deep Learning or Machine Learning? Leave a comment or ask your question via email . Will try my best to answer it.
Conclusion – We have seen in post above that ANN’s don’t create or invent any new information or facts. ANN help us make sense of what’s already in front of us hidden in our data.
Deep Learning in short is going much beyond machine learning and its algorithms that are either supervised or unsupervised. In DL it uses many layers of nonlinear processing units for feature extraction and transformation.
ANN’s structure is what enables artificial intelligence, machine learning and supercomputing to flourish. Neural networks are powers language translation, face recognition, picture captioning, text summarisation and lot more.
============================ About the Author =======================
Read about Author at : About Me
Thank you all, for spending your time reading this post. Please share your opinion / comments / critics / agreements or disagreement. Remark for more details about posts, subjects and relevance please read the disclaimer.
Categories: Neural Networks