What Is a Neural Network?

3 things you need to know

A neural network (also called an artificial neural network) is an adaptive system that learns by using interconnected nodes or neurons in a layered structure that resembles a human brain. A neural network can learn from data—so it can be trained to recognize patterns, classify data, and forecast future events.

A neural network breaks down the input into layers of abstraction. It can be trained using many examples to recognize patterns in speech or images, for example, just as the human brain does. Its behavior is defined by the way its individual elements are connected and by the strength, or weights, of those connections. These weights are automatically adjusted during training according to a specified learning rule until the artificial neural network performs the desired task correctly.

Why Do Neural Networks Matter?

Neural networks are especially well suited to perform pattern recognition to identify and classify objects or signals in speech, vision, and control systems. They can also be used for performing time-series prediction and modeling.

Here are a few examples of how artificial neural networks are used:

Deep Learning: Shallow and Deep Nets

Deep learning is a field that uses artificial neural networks very frequently. One common application is convolutional neural networks, which are used to classify images, video, text, or sound.

Neural networks that operate on two or three layers of connected neuron layers are known as shallow neural networks. Deep learning networks can have many layers, even hundreds. Both are machine learning techniques that learn directly from input data.

Deep learning is especially well suited to complex identification applications such as face recognition, text translation, and voice recognition. It’s also a key technology used in advanced driver assistance systems and tasks including lane classification and traffic sign recognition.

How Do Neural Networks Work?

A neural network combines several processing layers, using simple elements operating in parallel and inspired by biological nervous systems. It consists of an input layer, one or more hidden layers, and an output layer. In each layer there are several nodes, or neurons, with each layer using the output of the previous layer as its input, so neurons interconnect the different layers. Each neuron typically has weights that are adjusted during the learning process, and as the weight decreases or increases, it changes the strength of the signal of that neuron.

Typical neural network architecture.

Techniques Used with Neural Networks

Common machine learning techniques for designing artificial neural network applications include supervised and unsupervised learning, classification, regression, pattern recognition, and clustering.

Supervised Learning

Supervised neural networks are trained to produce desired outputs in response to sample inputs, making them particularly well suited for modeling and controlling dynamic systems, classifying noisy data, and predicting future events. Deep Learning Toolbox™ includes four types of supervised networks: feedforward, radial basis, dynamic, and learning vector quantization.

Classification

Classification is a type of supervised machine learning in which an algorithm “learns” to classify new observations from examples of labeled data.

Regression

Regression models describe the relationship between a response (output) variable and one or more predictor (input) variables.

Pattern Recognition

Pattern recognition is an important component of artificial neural network applications in computer vision, radar processing, speech recognition, and text classification. It works by classifying input data into objects or classes based on key features, using either supervised or unsupervised classification.

For example, in computer vision, supervised pattern recognition techniques are used for optical character recognition (OCR), face detection, face recognition, object detection, and object classification. In image processing and computer vision, unsupervised pattern recognition techniques are used for object detection and image segmentation.

Unsupervised Learning

Unsupervised neural networks are trained by letting the neural network continually adjust itself to new inputs. They are used to draw inferences from data sets consisting of input data without labeled responses. You can use them to discover natural distributions, categories, and category relationships within data.

Clustering

Clustering is an unsupervised learning approach in which artificial neural networks can be used for exploratory data analysis to find hidden patterns or groupings in data. This process involves grouping data by similarity. Applications for cluster analysis include gene sequence analysis, market research, and object recognition.

Developing Shallow Neural Networks with MATLAB

With tools and functions for managing large data sets, MATLAB® offers specialized toolboxes for working with machine learning, artificial neural networks, deep learning, computer vision, and automated driving.

With just a few lines of code, MATLAB lets you develop neural networks without being an expert. Get started quickly, create and visualize models, and deploy models to servers and embedded devices.

With MATLAB, you can integrate results into your existing applications. MATLAB automates deploying your artificial neural network models on enterprise systems, clusters, clouds, and embedded devices.

Typical Workflow for Designing Neural Networks

Each neural network application is unique, but developing the network typically follows these steps:

  1. Access and prepare your data
  2. Create the artificial neural network
  3. Configure the network’s inputs and outputs
  4. Tune the network parameters (the weights and biases) to optimize performance
  5. Train the network
  6. Validate the network’s results
  7. Integrate the network into a production system

Classification and Clustering of Shallow Networks

MATLAB and Deep Learning Toolbox provide command-line functions and apps for creating, training, and simulating shallow neural networks. The apps make it easy to develop neural networks for tasks such as classification, regression (including time-series regression), and clustering. After creating your networks in these tools, you can automatically generate MATLAB code to capture your work and automate tasks.

Preprocessing, Postprocessing, and Improving Your Network

Preprocessing the network inputs and targets improves the efficiency of shallow neural network training. Postprocessing enables detailed analysis of network performance. MATLAB and Simulink® provide tools to help you:

  • Reduce the dimensions of input vectors using principal component analysis
  • Perform regression analysis between the network response and the corresponding targets
  • Scale inputs and targets so they fall in the range [-1,1]
  • Normalize the mean and standard deviation of the training data set
  • Use automated data preprocessing and data division when creating your networks

Improving the network’s ability to generalize helps prevent overfitting, a common problem in artificial neural network design. Overfitting occurs when a network has memorized the training set but has not learned to generalize to new inputs. Overfitting produces a relatively small error on the training set but a much larger error when new data is presented to the network. Learn more about how you can use cross-validation to avoid overfitting.

Two solutions to improve generalization include:

  • Regularization modifies the network’s performance function (the measure of error that the training process minimizes). By including the sizes of the weights and biases, regularization produces a network that performs well with the training data and exhibits smoother behavior when presented with new data.
  • Early stopping uses two different data sets: the training set, to update the weights and biases, and the validation set, to stop training when the network begins to overfit the data

Postprocessing plots for analyzing network performance, including mean squared error validation performance for successive training epochs (top left), error histogram (top right), and confusion matrices (bottom) for training, validation, and test phases.

Applications Using Neural Networks

Lund University developed a predictive model and simulated thousands of risk-profile combinations to improve recipients' long-term survival rates.
Improving cancer diagnostics using a neural network to detect cancer from mass spectrometry data on protein profiles.
Train an artificial neural network to classify images of digits.