深度学习基本概念softmax、sigmoid、均方误差、交叉熵

在神经网络的最后一层总用到的激活函数为softmax和sigmoid。在经过以上激活函数后，线性回归选用均方误差为损失函数，逻辑回归选择交叉熵为损失函数。以下代码使用tensorflow实现。

softmax激活函数：用在多分类，一个输入只能有一个输出，输出标签独立（onehot类型标签，只有一个为1，其他都是0）），如猫狗大战一张图像，识别结果非猫即狗。

$Yn=\frac{e^{n}}{\sum_{i}^{k} e^{i}}$

tensorflow 实现 tf.nn.sigmoid(logits)，自己定义实现如下

def Sigmoid_( logits ):
    date = tf.cast(logits,tf.int32)
    result = []
    with tf.Session() as sess:
        date = sess.run(date)
        for i in date:
            sig = round(1/(1+math.exp(-i)),6 )
            result.append(sig)
    return result

sigmoid激活函数：用在多标签，即输入一张图像，可以同时多标签种类为1，如一张图像即有猫又有狗标签中含有两个1.

$Yn=\frac{1}{(1+e^{x})}$

tensorflow实现tf.nn.softmax(logits)，自定义实现

def Softmax_( logits ):
    date = tf.cast(logits,tf.int32)
    result = []
    with tf.Session() as sess:
        date = sess.run(date)
        sum = 0
        for i in date:
            sum += math.exp(i)
        for i in date:
            result.append( round(math.exp(i)/sum, 6))
        return result

softmax的交叉熵损失函数与sigmoid的略有不同，因为softmax标签只有一张可能性

softmax: $loss = -labels[i]*log(logits)$

sigmoid: $loss = -labels[i]*log(logits) - (1-labels[i])*log(1-logits)$

整体实现代码：

# coding:utf-8
import tensorflow as tf
import os
import math
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'


def Sigmoid_( logits ):
    date = tf.cast(logits,tf.int32)
    result = []
    with tf.Session() as sess:
        date = sess.run(date)
        for i in date:
            sig = round(1/(1+math.exp(-i)),6 )
            result.append(sig)
    return result


def Softmax_( logits ):
    date = tf.cast(logits,tf.int32)
    result = []
    with tf.Session() as sess:
        date = sess.run(date)
        sum = 0
        for i in date:
            sum += math.exp(i)
        for i in date:
            result.append( round(math.exp(i)/sum, 6))
        return result


def get_loss( res1 ):
    print("softmax loss")
    labels = [0, 1, 0]
    loss_sum = 0
    for i in range(len(labels)):
        loss_ = -labels[i]*tf.log(res1[i])
        loss_sum =loss_sum+loss_
    return  loss_sum


def get_sigmoid_loss( res1 ):
    print("sigmoid loss")
    labels = [0, 1, 0]
    loss_sum = []
    for i in range(len(labels)):
        loss_ = -labels[i]*tf.log(res1[i]) - (1-labels[i])*tf.log(1-res1[i])
        loss_sum.append(loss_)
    return  loss_sum

print('=====================sigmod test===============================')

logits = tf.constant([2,5,1],dtype=tf.float32)
labels = [0,1,0]
label = tf.convert_to_tensor(labels)
label = tf.cast(label,tf.float32)

res_sigmoid = tf.nn.sigmoid(logits)

ls = get_sigmoid_loss(res_sigmoid)
res2 = Sigmoid_(logits)
res3 = tf.nn.sigmoid_cross_entropy_with_logits(logits=logits,labels=label)

with tf.Session() as sess:
    res1,res3 = sess.run([res_sigmoid,res3])
    los = sess.run(ls)
    print("tf计算结果：",   res1)
    print("手写计算结果：",  res2)
    print("tf计算结果：",   res3)
    print("手写计算结果：",  los)

print('====================softmax test================================')
res1 = tf.nn.softmax(logits)
ls = get_loss(res1)

res2 = Softmax_(logits)
res3 = tf.nn.softmax_cross_entropy_with_logits_v2(logits=logits,labels=labels)
with tf.Session() as sess:
    res1,res3 = sess.run([res1,res3])
    los = sess.run(ls)
    print("tf计算结果：",   res1)
    print("手写计算结果：", res2)
    print("tf计算结果：",   res3)
    print("手写计算结果：",  los)

print('====================================================')

不你不想

发布了19 篇原创文章 · 获赞 28 · 访问量 1万+

私信关注

深度学习基本概念softmax、sigmoid、均方误差、交叉熵

猜你喜欢