Similarities and differences between sigmoid and tanh
Calculation process (if wrong, please correct):
1. sigmoid function
official:
Domain: R
Value range: (0,1)
Derivative:
Derivative function domain: R
Derivative function range: (0,0.25]
image:
2.tanh function
official:
Domain: R
Value range: (-1,1)
Derivative:
Derivative function domain: R
Derivative function range: (0,1]
image:
They are all information transfer functions, or activation functions, in artificial neural networks.
The relationship between the two functions:
From the value range of their functions and derivative functions, we can see why they are selected as activation functions.
subtle differences between them
Observe the function curves of sigmoid and tanh. When the input of sigmoid is between [-1, 1], the function value is sensitive to changes. Once it approaches or exceeds the interval, it loses its sensitivity and is in a saturated state, which affects the accuracy of neural network prediction. The output and input of tanh can maintain a nonlinear monotonous rise and fall relationship, which is in line with the gradient solution of the BP network, with good fault tolerance, bounded, and asymptotic to 0 and 1, which is in line with the law of human brain nerve saturation, but it is more saturated than the sigmoid function. Expect.
Other functions that satisfy the domain of R and the range of (0,1)
References:
Wikipedia link for Sigmoid https://en.wikipedia.org/wiki/Sigmoid_function
Tanh's Wikipedia link https://en.wikipedia.org/wiki/Hyperbolic_function
http://blog.sina.com.cn/s/blog_6bb5e91b0102vbbr.html