## Feed Forward Neural Networks

In a feed forward neural network, neurons cannot form a cycle. In this episode, we explore how such a network would be able to represent three common logical operators: OR, AND, and XOR. The XOR operation is the interesting case.

Below are the truth tables that describe each of these functions.

### AND Truth Table

Input 1 Input 2 Output
0 0 0
0 1 0
1 0 0
1 1 1

### OR Truth Table

Input 1 Input 2 Output
0 0 0
0 1 1
1 0 1
1 1 1

### XOR Truth Table

Input 1 Input 2 Output
0 0 0
0 1 1
1 0 1
1 1 0

The AND and OR functions should seem very intuitive. Exclusive or (XOR) if true if and only if exactly single input is 1. Could a neural network learn these mathematical functions?

Let's consider the perceptron described below. First we see the visual representation, then the Activation function , followed by the formula for calculating the output.

Can this perceptron learn the AND function?

Sure. Let and

Yup. Let and

An infinite number of possible solutions exist, I just picked values that hopefully seem intuitive. This is also a good example of why the bias term is important. Without it, the AND function could not be represented.