Use matplitilb python painting neural network image activation function

1.sigmoid

import numpy as np
from matplotlib import pyplot as plt

# sigmoid
x = np.linspace(-7, 7, 50)
y = 1 /(1 + np.exp(-x))
plt.figure(1)
plt.xlabel("x")
plt.ylabel("y")
plt.ylim(-0.1, 1.1)
plt.title(r"$f(x) = 1 /(1 + e^{-x})$")
plt.plot(x,y)
plt.grid()
plt.show()

2.tanh

# tanh
x = np.linspace(-7, 7, 50)
y = (1 - np.exp(-2 * x)) /(1 + np.exp(-2 * x))
plt.figure(1)
plt.xlabel("x")
plt.ylabel("y")
plt.ylim(-1.1, 1.1)
plt.title(r"$f(x) = (1 - e^{-2x}) /(1 + e^{-2x})$")
plt.plot(x,y)
plt.grid()
plt.show()

  If the sigmoid and tanh function graph drawing together and you will find them very similar shape, and then observe their functional, you will find, in fact:

 tanh = 2sigmoid(2x)-1

  Derivation hands might look

3.softplus and relu

# relu和softplus画在一张图上
x = np.linspace(-5, 5, 50)
y1 = [max([0, xpoint]) for xpoint in x]
y2 = np.log(1 + np.exp(x))
plt.figure(1)
plt.xlabel("x")
plt.ylabel("y")
plt.title(r"$f(x) = relu(x)$")
plt.plot(x,y1)
plt.plot(x,y2)
plt.grid()
plt.show()