In this tutorial, we will create a single layer perceptron. In this sensor model, we will create, we will try to train our network by using forward and backward propagation algorithms.


Perceptron


The simplest single layer neural network model is Perceptron. It consists of the input and output layer. It is defined as the smallest learning unit of artificial neural networks. It is a feed forward and supervised algorithm. In the feed forward neural network, inputs are transmitted unidirectionally from the input layer to the output layer. In supervised learning, the difference between outputs and goals is considered as error. In order to minimize this error, the weight values should be trained, ie updated appropriately. For each sample, both inputs and outputs that need to be generated in return for these inputs are specified on the network.


How to Learn a Perceptron?


In the Perceptron learning algorithm, an initial value is first assigned to all weights and a threshold value and learning coefficient are determined. Then the inputs are given and a result is obtained by multiplying the inputs by their weights and adding them by the bias value. If this result is greater than the specified threshold, 1 is returned, and 0 is returned if it is less. If the calculated value and the expected value are not the same, the difference between them is taken and the weight value is updated according to this value. This process is continued until the desired result is achieved. Thus, the learning is completed.


Activation Fuction


We need the activation function to introduce nonlinear real world features to artificial neural networks. Basically, in a simple neural network, x is defined as inputs, w weights, and we apply f (x), or activation, to the value transferred to the output of the network. Then this will be the final exit or the entrance to another layer. Activation functions allow us to bring our outputs to a certain range, preventing our output from reaching very large sizes. The most well-known activation functions are sigmoid, tanh, relu, etc.


How to create a single layer Perceptron?


As the name suggests, a single layer artificial neural network model has one layer. In artificial neural network models, the input layer is not considered as a layer. The single layer perceptron we will create is shown in the image below.


FeedForward Propagation


hand_neural_networks_1.JPG

Step 1: We multiply the weight values by the input values and then add them.

z = x1 * w1 + x2 * w2

forward_step_1.JPG

Step 2: To find the value of a, we put the z value into the sigmoid function. Sigmoid function is one of the activation functions.

a = sigmoid (z)

sigmoid = 1 / (1 + e ^ -z)


forward_step_2.JPG

Step 3: The value a we found in the second step also belongs to the y_head value. At this stage we will find the error value.

y_head = a

error = 1/2(y – y_head)^2

forward_step_3.JPG

Backward Propagation


The back propagation algorithm is more difficult to understand than the forward propagation algorithm. In this backpropagation algorithm, we are trying to update the weight values of our artificial neural network. To update these weight values, we get derivative our output functions. We use the partial derivative method for this. If you do not know about derivatives, you can get information about derivatives by using this link.


Step 1: To find the smallest change in the value of w1, we need to find the derivative of the error function with respect to w1. But since our error function does not contain the value of w1, we need to derive the part.

chain_ruler.JPG

Step 2: To find the derivative of the error respect to w1, we must multiply the values.


dEdivdw1.JPG

Step 3: To find the smallest change in the value of w2, we need to find the derivative of the error function with respect to w2. But since our error function does not contain the value of w2, we need to derive the part.

chain_ruler2.JPG

Step 4: To find the derivative of the error respect to w2, we must multiply the values.


dEdivdw2.JPG

Step 5: To find the new values ​​of the weights, we must subtract the new found weight value from the old weight value.

w1 = old_w1 – derivative_w1

w2 = old_w2 – derivative_w2

update_weights.JPG

In the next tutorial we will see how to create a single layer neural network with coding.