Feedforward Neural Network
Feedforward Neural Network (FNN) is a type of artificial neural network in which information flows in a single direction i.e from the input layer through hidden layers to the output layer without loops or feedback. It is mainly used for pattern recognition tasks like image and speech classification.
For example in a credit scoring system, banks use an FNN which analyze users financial profiles such as income, credit history and spending habits to determine their creditworthiness.
Each piece of information flows through the network’s layers where various calculations are made to produce a final score.
Structure of a Feedforward Neural Network
Feedforward Neural Networks have a structured layered design where data flows sequentially through each layer.
Input Layer: The input layer consists of neurons that receive the input data. Each neuron in the input layer represents a feature of the input data.
Hidden Layers: One or more hidden layers are placed between the input and output layers. These layers are responsible for learning the complex patterns in the data. Each neuron in a hidden layer applies a weighted sum of inputs followed by a non-linear activation function.
Output Layer: The output layer provides the final output of the network. The number of neurons in this layer corresponds to the number of classes in a classification problem or the number of outputs in a regression problem.
Activation Functions
Activation functions introduce non-linearity into the network enabling it to learn and model complex data patterns.
Common activation functions include:
- Sigmoid:
- Tanh:
- ReLU:
Training a Feedforward Neural Network
Training a Feedforward Neural Network involves adjusting the weights of the neurons to minimize the error between the predicted output and the actual output. This process is typically performed using backpropagation and gradient descent.
Forward Propagation: During forward propagation the input data passes through the network and the output is calculated.
Loss Calculation: The loss (or error) is calculated using a loss function such as Mean Squared Error (MSE) for regression tasks or Cross-Entropy Loss for classification tasks.Backpropagation: In backpropagation the error is propagated back through the network to update the weights. The gradient of the loss function with respect to each weight is calculated and the weights are adjusted using gradient descent.