ELU activation function

Others 2019-06-25 12:51:22 views: null

ELU activation function

Write pictures described here

Fusion and RELU sigmoid, left soft-saturation, right no unsaturation.
ELU linear portion such that the right side can be alleviated gradient disappears, while the left soft ELU enables a more robust saturation change of the input or noise.
ELU Mean output is approximately zero, therefore faster convergence.
On ImageNet, without Batch Normalization more than 30 layers ReLU network will not converge, PReLU network in the MSRA's Fan-in (caffe) initialization will diverge, and ELU network can converge in the Fan-in / Fan-out.

https://blog.csdn.net/zrh_CSDN/article/details/81266188

Guess you like

Origin blog.csdn.net/qq_34106574/article/details/89944691

ELU activation function

[Activation Function] Several activation functions you must know in deep learning: Sigmoid, Tanh, ReLU, LeakyReLU and ELU activation functions (latest compiled in 2024)

Activation function summary (sigmoid, tanh, RELU, LRel, PReLU, ELU, GELU and SELUS)

YOLOv5 improvement series (13) - replace the activation function of SiLU, ReLU, ELU, Hardswish, Mish, Softplus, AconC series, etc.

Activation function Activation Function

Activation function

[Activation function] PReLU activation function

[Activation function] SELU activation function

[Activation function] GELU activation function

On activation function of neural networks

Why introduce the activation function?

What is the sigmoid activation function?

pytorch 之 activation function

SOTA activation function learning

CNN- activation function

Relu advantage activation function

Depth study: the activation function

Keras activation function difference

Detailed activation function Mish

neural network activation function

Gradient of activation function and Loss

PyTorch - activation function

Saturated (unsaturated) activation function

activation function 0

A survey of the activation function and loss of function

What is the incentive function (Activation Function)

Activation Function and Ten Common Activation Functions

Activation function, loss function, optimization function

Assumption function, activation function, output function

The difference between the Sigmoid activation function and the ReLU activation function:

Recommended

Ranking

Empire cms smart tag calls four first-level recommended articles, starting from the fourth article

Linux environment installation and configuration Elasticsearch7.17

Big Data processing architecture and Lambda Kappa architecture

Explore the top of the AI large model platform - Wenxin Qianfan

Beijing car PK10 lucky airship Guanya size and value of the odd and even tips

W3B x Sui Hacker House｜In-depth understanding of Sui and Move language

Know almost Ko Chan: Chinese what any decent open source software products? (Finishing from my original answer)

Comprehensively improve AD domain security authentication | Zhuyun IDaaS

Android Update Engine Analysis (24) What happened when making the downgrade package?

Spark Architecture and Operating Mechanism (1) - System Architecture

Daily

More

2024-05-07(34)

2024-05-06(6)

2024-05-05(0)

2024-05-04(18)

2024-05-03(8)

2024-05-02(0)

2024-05-01(4)

2024-04-30(36)

2024-04-29(5)

2024-04-28(12)