Deep learning 1 perceptron and implementation of simple back propagation network

What is a perceptron ？
The sensor receives multiple input signals , Output a signal
The sensor's signal is only “ flow / No flow ” Two values ,0 Corresponding “ No signal transmission ”,1 Corresponding “ Signal transmission ”.
Input signals are sent to neurons , Will be multiplied by a fixed weight . The neurons calculate the sum of the signals they send , Only when the sum exceeds a certain threshold , Will be output 1. This is also called “ Neurons are activated ”.
The above contents are expressed by mathematical formula , Namely ：
y=0（w1x1+w2x2<=θ）
1 (w1x1+w2x2>θ)

Simple logic circuit
And gate and gate: And gate is a gate circuit with two outputs and one input . And gate only when both inputs are 1 Time output 1, Output at other times 0
NAND gate NAND gate
Or gate

The implementation of perceptron
Implementation of and gate

def AND(x1,x2):
    w1,w2,theta = 0.5,0.5,0.7
    tmp = w1*x1+w2*x2
    if tmp <= theta:
        return 0
    elif tmp > theta：
        return 1

Import weights and offsets
y=0(b+w1x1+w2x2<=0)
1(b+w1x1+w2x2>0)
The value of bias determines how easily neurons are activated

Limitations of perceptron
Exclusive OR gate , Only when the x1 and x2 One of them is 1 when , Will be output 1.
A straight line cannot be separated In truth table 4 A little bit

The space divided by curves is called nonlinear space , The space divided by straight lines is called linear space .

Multilayer perceptron

The implementation of XOR gates
AND function .NAND function .OR function . superposition

def XOR(x1, x2):
    s1= NAND(x1,x2)
    s2= OR(x1,x2)
    y=AND(s1,s2)
    return y

Back propagation network

# Define activation function 
import numpy as np
def sigmoid(x, deriv=Flase)
    if deriv==True:
        return x*(1-x)
    return 1/(1+np.exp(-x))
# Network structure 
w0 = np.random.random(( , ))
w1 =  np.random.random(( , ))
# Forward propagation 
l0 = x
l1 = sigmoid(np.dot(l0, w0))
l2 = sigmoid(np.dot(l1,w1))
l2_error = l2 - y
# Back propagation 
l2_delta = l2_error * sigmode(l2, deriv=True)
l1_error = np.dot(l2_delta, w1.T)
l1_delta = l1_error*sigmode(l1, deriv=True)
# Update parameters 
w1-=l1.T .dot( l2_delta)
w0-=l0.T .dot( l1_delta)