Neural Network Models in R

In this tutorial, you will learn how to create a Neural Network model in R.

Neural Network(or Artificial Neural Network) has the ability to learn by examples. ANN is an information processing model inspired by the biological neuron system. It is composed of a large number of highly interconnected processing elements known as the neuron to solve problems. It follows the non-linear path and process information in parallel throughout the nodes. A neural network is a complex adaptive system. Adaptive means it has the ability to change its internal structure by adjusting the weights of inputs. (source)

The neural network was designed to solve problems that are easy for humans and difficult for machines such as identifying pictures of cat and dog, identifying numerical number pictures. such problems often referred to as pattern recognition. Its application ranges from optical character recognition to object detection.

In this tutorial, you are going to cover the following topics:

  • Introduction to neural network
  • Forward Propagation and Back Propagation
  • Activation Function
  • Implementation of the neural network in R
  • Use-cases of NN
  • Pros and Cons
  • Conclusion

For more such tutorials, projects, and courses visit DataCamp:

Introduction to Neural Network

In 1943, Warren McCulloch and Walter Pitts developed the first mathematical model of a neuron. In their research paper “A logical calculus of the ideas immanent in nervous activity”, they described the simple mathematical model for a neuron, which represents a single cell of the neural system that takes inputs, processes those inputs, and returns an output. This model is known as the McCulloch-Pitts neural model. source

NN algorithms are inspired by the human brain to perform a particular task or function. NN performs computations through a process by learning. The neural network is a set of connected input/output units in which each connection has a weight associated with it. In the learning phase, the network learns by adjusting the weights to predict the correct class label of the given inputs.

The human brain consists of billions of neural cells that process information. Each neural cell considered a simple processing system. The Interconnected Web of neurons known as a biological neural network transmits information through electrical signals. this parallel interactive system makes the brain to think and process information. Dendrites of a neuron receive input signals from another neuron and respond to output based on those inputs to an axon of some other neuron. and, based on those inputs, fire an output signal via an axon.

Image Source: cs231n.github.io

Dendrites receive signals from other neurons. The cell body sums all the inputs signals to generate output. Axon through output When the sum reaches a threshold. Synapse is a point of interaction between neurons. It transmits electrical or chemical signals to another neuron. Synapse is derived from the Greek word which means conjunction.

Source

Here, x1,x2….xn are input variables. w1,w2….wn are weights of respective inputs. b is the bias, which is summed with the weighted inputs to form the net inputs. bias and weights are both adjustable parameters of the neuron. Parameters are adjusted using some learning rules. The output of a neuron can range from -inf to +inf. The neuron doesn’t know the boundary. So we need a mapping mechanism between the input and output of the neuron. This mechanism of mapping inputs to output is known as Activation Function.

Feedforward and Feedback Artificial Neural Networks

There are two main types of artificial neural networks: Feedforward and feedback artificial neural networks. Feedforward neural network is a network that is not recursive. neurons in this layer were only connected to neurons in the next layer and they are don’t form a cycle. In Feedforward signals travel in only one direction towards the output layer (source)

Feedback neural networks contain cycles. Signals travel in both directions by introducing loops in the network. The feedback cycles can cause the network’s behavior to change over time based on its input. Feedback neural network is also known as recurrent neural networks.

Source

Activation Functions

The activation function defines the output of a neuron in terms of the induced local field. Activation functions are a single line of code that gives the neural nets non-linearity and expressiveness. There are many activation functions. Some of them are as follows:

  • The identity function is a function that maps input to the same output value. It is a linear operator in vector space. Also, a known straight-line function where activation is proportional to the input.
  • In Binary Step Function, if the value of Y is above a certain value known as the threshold, the output is True(or activated) and if it’s less than the threshold then the output is false (or not activated). It is very useful in the classifier.
  • Sigmoid Function called S-shaped functions. Logistic and hyperbolic tangent functions are commonly used in sigmoid functions. There are two types of sigmoid functions.
  • Binary Sigmoid Function is a logistic function where the output values are either binary or vary from 0 to 1.
  • Bipolar Sigmoid Function is a logistic function where the output value varies from -1 to 1. Also known as Hyperbolic Tangent Function or tanh.
  • Ramp Function: The name of the ramp function is derived from the appearance of its graph. It maps negative inputs to 0 and positive inputs to the same output.
  • ReLu stands for the rectified linear unit (ReLU). It is the most used activation function in the world. It output 0 for negative values of x.

Implementation of Neural Network in R

Install the required package

Let’s first install neuralnet library

# install packageinstall.packages("neuralnet")Updating HTML index of packages in '.Library'
Making 'packages.html' ... done

Create training dataset

Let’s create your own dataset. Here you need two kinds of attributes or columns in your data: Feature and label. I above shown table you can see the technical knowledge, communication skills score, and placement status of students. So the first two columns(Technical Knowledge Score and Communication Skills Score) are features and the third column(Student Placed) is the binary label.

# creating training data setTKS=c(20,10,30,20,80,30)CSS=c(90,20,40,50,50,80)Placed=c(1,0,0,0,1,1)# Here, you will combine multiple columns or features into a single set of datadf=data.frame(TKS,CSS,Placed)

Let’s build the NN classifier model using neuralnet library.

First, import the neuralnet library and create the NN classifier model by passing an argument set of labels and features, dataset, number of neurons in hidden layers, and error calculation.

# load libraryrequire(neuralnet)# fit neural networknn=neuralnet(Placed~TKS+CSS,data=df, hidden=3,act.fct = "logistic", linear.output = FALSE)

Here,

- Placed~TKS+CSS, Placed is label annd TKS and CSS are features.
- df is dataframe,
- hidden=3: represents single layer with 3 neurons respectively.
- act.fct = "logistic" used for smoothing the result.
- linear.ouput=FALSE: set FALSE for apply act.fct otherwise TRUE

Plotting Neural Network

Let’s plot your neural net model.

# plot neural networkplot(nn)

Create test dataset

Create test dataset using two features Technical Knowledge Score and Communication Skills Score

# creating test setTKS=c(30,40,85)CSS=c(85,50,40)test=data.frame(TKS,CSS)

Predict the results for test set

Predict the probability score for test data using the compute function.

## Prediction using neural networkPredict=compute(nn,test)Predict$net.result
Output:
0.9928202080 0.3335543925 0.9775153014

Now, Convert probabilities into binary classes.

# Converting probabilities into binary classes setting threshold level 0.5prob <- Predict$net.resultpred <- ifelse(prob>0.5, 1, 0)pred
Output:
101

Predicted results are 1,0, and 1.

Pros and Cons

Neural networks are more flexible and can be used with both regression and classification problems. Neural networks are good for the nonlinear dataset with a large number of inputs such as images. Neural networks can work with any number of inputs and layers. Neural networks have the numerical strength that can perform jobs in parallel.

There are more alternative algorithms such as SVM, Decision Tree, and Regression that are available that are simple, fast, easy to train and provide better performance. Neural networks are much more of the black box, require more time for development and more computation power. Neural Networks requires more data than other Machine Learning algorithms. NNs can be used only with numerical inputs and non-missing value datasets. A well-known neural network researcher said “A neural network is a second-best way to solve any problem. The best way is to actually understand the problem,”

Use-cases of NN

NN’s wonderful properties offer many applications such as:

  • Pattern Recognition: neural networks are very suitable for pattern recognition problems such as facial recognition, object detection, fingerprint recognition, etc.
  • Anomaly Detection: neural networks are good at pattern detection and they can easily detect the unusual patterns that don’t fit in the general patterns.
  • Time Series Prediction: Neural networks can be used to predict time series problems such as stock price, weather forecasting.
  • Natural Language Processing: Neural networks offer a wide range of applications in Natural Language Processing tasks such as text classification, Named Entity Recognition (NER), Part-of-Speech Tagging, Speech Recognition, and Spell Checking.

Conclusion

Congratulations, you have made it to the end of this tutorial!

In this tutorial, you have covered a lot of details about the Neural Network. you have learned what is Neural Network, Forward Propagation, and Back Propagation, Activation Functions, Implementation of the neural network in R, Use-cases of NN, Pros, and Cons of NN.

Hopefully, you can now utilize the Neural Network concept to analyze your own datasets. Thanks for reading this tutorial!

Originally published at https://www.datacamp.com/community/tutorials/random-forests-classifier-python
Do you want to learn data science, check out on DataCamp.

For more such article, you can visit my blog Machine Learning Geek

Reach out to me on Linkedin: https://www.linkedin.com/in/avinash-navlani/

Sr Data Scientist| Analytics Consulting | Data Science Communicator | Helping Clients to Improve Products & Services with Data

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store