Artificial Neural Networks – As the name suggest “Neural Network”, they are inspired by the human brain system. As ANNs were originally designed and inspired with biological neurons thus some time they are called as brain model for computers. They process information in form audio, video, images, texts and numbers. Neurons i.e. perceptrons (as known in early days) were staged as a decision function in those times. ANNs are designed to take several binary inputs to give a binary output. Professor Frank Rosenblatt was the first one to use neural networks.
Artificial Neural Networks (ANN) – Some background
As per Dr Robert Hecht-Nielsen, the inventor of one of the first neuron computer, ANN is “A computing system made up of a number of simple, highly connected processing elements, which process information by their dynamic state response to external inputs.”
AILabPage defines – Artificial neural networks (ANNs) as “A complex computer code written with a number of simple, highly interconnected processing elements which is inspired by human biological brain structure for simulating human brain working & processing data (Information) models“. Point to note, it’s way different than traditional computer program though.
Human brain needs much smaller data set when processing or classifying data sets as compared to computer simulation (neural networks) to learn patterns and captioning images. Human brain so powerful and step ahead of super fast terminology when we need to identify images, handwriting, speaking different languages and watching videos to understand & re-create them. For machine same skills are very complex but ANNs are making it possible now.
This post is just an overview of artificial neural networks. Idea is to explain what is neural network, what it can do and how to use them in our machine learning challenges. This post is written in simple english for readers who are new to this animal. Phd scholars, professionals with experience are welcome to comment to improve this simple english paper.
Artificial Neural Networks (ANN) – Building Blocks
In 1943 Warren S. McCulloch and Walter Pitts came out with a highly simplified model of a neuron in their paper and made important contribution to the development of artificial neural networks. ANNs consist of several parameters, hyper-parameters and layers. Parameters and hyper-parameters helps to drive output from neural network model. Weights, number of epochs, biases, batch size and learning rates are few examples of parameters. Neural nets consist of 3 layers. These nodes process information:
- Input Layers: The entry point to neural network, this entry point or layer takes input data as numbers, texts, audio, videos and image as pixels etc.
- Hidden Layers: The next layer or input taker from the output of input layer. This layer is responsible for number crunching i.e. mathematical operation, paying attention to detect patterns data which human eye cant see and extractor of feature etc. There can be minimum of one and many multiple hidden layers.
- Output Layer: Takes input from hidden layer to generate the desired output.
Artificial neural networks has very important role for building deep learning models. Like human brain ANN get formed of neuron to process information. ANNs has excellent possibility & flexibility to model, establish and ease out complex relationships between input and output. The neural networks are now part of research subject like Neuro-informatics (a research field concerned with the organisation of neuroscience data by the application of computational models and analytical tools).
Artificial Neural Networks (ANN) – Answer to Many Questions
Artificial neural network concept came out of human brain working model as an inspiration to allow machines to read & generate pictures, texts for translations, videos and sounds. Today we have fully working models of neural networks which can answer question through voice processing, recognising handwritings, watch and interpret videos, understand and predict our behaviour.
It can alert security officials and send location of the wanted person through face detection and body language behaviour. Neural networks are always at foundation level for many of the biggest break through in artificial intelligence. Yes also can drive us around in our cars
Neural networks use supervised as well as unsupervised machine learning data sets and methodologies to achieve results. Supervised learning method is much more suitable and appropriate when neural networks are used for classification problem like email a spam or not spam. Often people get confused between deep neural network which is actually an artificial neural network only with multiple layers between the input and output layers.
Question here can be how neural network manage to do so? Well answer is simple and complex at same time as it all boils down to the data we generate. These networks reads, understand and find patterns in our data to learn and fine-tune it self. More the data it process more smartness it acquires over time. It requires extensive training of neural network before implementing them in real-time problem solving environment, So neural networks can
- Predict our behavior and become expert.
- Learn to understand and speak our language, text and hand writing
- Watch videos to learn facial movement for pronouncing different words and create videos
Neural networks they can solve almost any problem. Problems like identifying signatures on bank cheques, cluttering spoken words of different languages, credit card fraudulent transactions, and even unwanted email. Point to note here is neural networks are very effective, powerful and excellent tool to find output for any issue but keep in mind that just because you have a new hammer, it doesn’t mean that everything is a nail (highlighted text taken from internet). We need to understand, design on paper and strategies our path for solution to a problem in hand.
Artificial Neural Networks – Types and Kinds
ANN’s learns, get trained and adjust automatically like we human do. Though ANN’s are inspired by human brain but for a fact they run on a far simpler plane. The structure of neurons are now used for machine learning thus called as artificial learning. This development has helped various problems to come to an end especially where layering is needed for refinement and granular details are needed.
There are several kinds of Neural Networks in deep learning. Which one needs to be picked is entirely depends upon the data to train and end goal. Voice recognition kind of challenges might need combinations. Neural networks consist of input and output layers and at-least one hidden layer.
- Multi-Layer Perceptron
- Radial Basis Network
- Hopfield Networks
- Boltzmann Machine Networks
- Recurrent Neural Networks
- Generative Adversarial Networks
- Convolutional Neural Networks.
The most widely used type of neural networks are RNN- Recurrent Neural Networks. In RNNs data flow is in multiple directions. These networks are employed for highly complex tasks i.e voice recognition, handwriting and language recognition etc. Thus RNNSs abilities are quite a limitless.
Neural network based on radial basis function can be used for strategic reasons. There are several other models of neural network including what we have mentioned above. For an introduction to neural network and their working model continue reading this post. You will get a sense of how they work and used for real mathematical problems.
In short ANN’s are designed to simulate computer’s working process the way the human brain processes information. ANN’s got popular just in recent times as in past we neither had the computing power nor the required amount of data to train them.
Artificial Neural Networks Components
Neural Networks work as a visual guide for data evolution strategies which are highly scalable alternative for deep reinforcement learning. Deep neural networks are fed through raw data; which learns to identify the object on which it is trained or “being trained”. From the architectural point of view artificial neural networks has three components.
- Model Topology (Connections) – To describes the layers of neurons and structure of the connections between them.
- Activation Function (Transfer Function) – Function to be used by the artificial neurons.
- Learning Algorithm – To find the ideal values of the weights.
Each “neuron” is a relatively simple element e.g. summing its inputs and applying a threshold to the result, to decide the output of that “neuron”. Deep Neural Networks are the first family of algorithms within machine learning that do not require manual feature engineering, rather, they learn on their own by processing and learning the high-level features from raw data.
The data thrown into neural network reveal some pattern (hopefully). On the down side, neural network works like complete mystery/black box. Neural network can find patterns in data but how and why it exist, never tells this secrete so because it cant explain such matter thus we need data scientist to do this job.
Neural Network Working
Deep Learning uses neural networks to create its foundation of working model architecture. Neural network which took idea of human brain working for its basic working model has its basic unit a neuron. Like human brain artificial neural network mimic the similar information processing model i.e.:
- Like brain sees some pictures through eyes and combine the feature of the picture, similarly neural network takes input as linear combinations through input neurons and combine it for processing.
- Brain does something and give output without efforts similarly ANNs does some processing and give output. In ANNs it can show and detect that there are some patterns and useful information but cant tell how and why.
At time of inputs in form of linear combination, we need to apply some function for optimisation reasons. This function is called as the activation function to achieve desired output. Some common examples are:
- Sigmoid Function – A function to crushes all inputs to allow output output to be between 0 and 1
- ReLU Function – The Rectified Linear Unit function is much better function then tanh function and also the most commonly used activation function. Returns 0 for negative inputs and stay silent for all positive values outputs.
- Tanh Function – A function to crushes all inputs to allow output output to be between -1 and 1.
After making explanation above now we can say on line, “Neural network is a complicated formation of neuron like structures”. It has excellent possibility & flexibility to model, establish and ease out complex relationships between input and output. Hidden remains hidden though.
Deep Learning or Machine Learning
Making decision or choosing between ML or DL depends mainly on the problem & data at hand. Some high level differences as below. Should you choose to read in details then I suggest you read “Demystifying the difference between Machine Learning and Deep Learning“.
- ANN’s – In ANN’s learning is based on multiple layers of features or representation in each layer with the layers forming a hierarchy of low-level to high-level features. It may sound far-fetched, but remember that humans are neurologically hard-wired. In ANN’s Focus is really on end-to-end automatic learning based on raw features.
- Machine learning – In ML focus is on feature engineering. Traditional machine learning create / train test splits of the data where ever possible via cross-validation.
In a scenario where deep understanding of medical symptoms is required for some disease detection, a high performance and accuracy is very crucial. So in this case deep learning would be a better choice compare to traditional machine learning.
Frequently used Jargons in Deep Learning
- Perceptrons – A single layer neural network. Perceptron is a linear classifier. It is used in supervised learning. In this computing structures are based on the design of the human brain and algorithms takes a set of inputs and returns a set of outputs.
- Multilayer Perceptron (MLP)- A Multilayer Perceptron is a Feedforward Neural Network with multiple fully-connected layers that use nonlinear activation functions to deal with data which is not linearly separable.
- Deep Belief Network (DBN) – DBN is a type of probabilistic graphical model that learn a hierarchical representation of the data in an unsupervised manner.
- Deep Dream – A technique invented by Google that tries to distil the knowledge captured by a deep Convolutional Neural Network.
- Deep Reinforcement Learning (DRN) – This is a powerful and exciting area of AI research, with potential applicability to a variety of problem areas. Other common terms under this area are DQN, Deep Deterministic Policy Gradients (DDPG) etc.
- Deep Neural Network (DNN) – A neural network with many hidden layers. There is no hard-coded definition of how many layers minimum a deep neural network has to have. Usually minimum 3-5 or more.
- Recursive Neural Networks – Recursive Neural Networks – Call it as a deep tree-like structure. When the need is to parse a whole sentence we use the recursive neural network. Tree-like topology allows branching connections and hierarchical structure. Arguments here can be how recursive neural network are different then recurrent neural networks?
- Recurrent Neural Networks – RNNs abilities are quite a limitless. Don’t get lost between Recursive and Recurrent NNs. ANN’s structure is what enables artificial intelligence, machine learning and supercomputing to flourish. Neural networks are used for language translation, face recognition, picture captioning, text summarization and lot more tasks.
- Convolutional Neural Networks – Convolutional Neural Networks (CNN) are an excellent tool and one of the most advanced achievements in deep learning. In today’s time, CNN’s deep learning got hyped, much of attention and focus from all players in the business. The two core concepts of convolutional neural networks are convolution (hence the name) and pooling.
- Generative Adversarial Networks – GANs are a class of unsupervised learning algorithms used environment. As the name suggests they are called as Adversarial Networks because they are is made up of two competing neural networks i.e Generator and Discriminator, Both networks compete with each other to achieve zero-sum game. Both neural networks are assigned different job role i.e. contesting with each other:
Points not covered here
- Adding weights
- Sigmoid neuron/functions
- How to give the neurons an activation bias
- Determine the activation level
Points to Note:
When to use artificial neural networks as oppose to traditional machine learning algorithms is a complex one to answer. It entirely depends upon on the problem in hand to solve. One needs to be patient and experienced enough to have the correct answer. All credits if any remains on the original contributor only. In the next upcoming post will talk about Recurrent Neural Networks in detail.
Books Referred & Other material referred
- Open Internet research and reading
- AILabPage (group of self-taught engineers) members hands-on lab work.
- Learning to live webinars and conferences attendance
Feedback & Further Question
Do you have any questions about Deep Learning or Machine Learning? Leave a comment or ask your question via email . Will try my best to answer it.
Conclusion – We have seen in the post above that ANN’s don’t create or invent any new information or facts. ANN help us to make sense of what’s already available in the hidden format. It takes an empirical approach on a massive amount of data to give best and near accurate results. Deep Learning, in short, is going much beyond machine learning and its algorithms that are either supervised or unsupervised. In DL it uses many layers of nonlinear processing units for feature extraction and transformation.
============================ About the Author =======================
Read about Author at : About Me
Thank you all, for spending your time reading this post. Please share your opinion / comments / critics / agreements or disagreement. Remark for more details about posts, subjects and relevance please read the disclaimer.
Categories: Neural Networks