TensorFlow中常用的损失函数

TensorFlow的tf.nn模块里的损失函数：

tf.nn.softmax_cross_entropy_with_logits_v2()

import tensorflow as tf labels = [[0.2,0.3,0.5], [0.1,0.6,0.3]] logits = [[2,0.5,1], [0.1,1,3]] logits_scaled = tf.nn.softmax(logits) result1 = tf.nn.softmax_cross_entropy_with_logits_v2(labels=labels, logits=logits) # 结果正确 result2 = -tf.reduce_sum(labels*tf.log(logits_scaled),1) # 根据交叉熵损失函数的定义计算,结果正确 result3 = tf.nn.softmax_cross_entropy_with_logits_v2(labels=labels, logits=logits_scaled) # 结果不合预期,用法不对,输入不应当是尺度化之后的概率 with tf.Session() as sess: print(sess.run(result1)) print(sess.run(result2)) print(sess.run(result3)) ''' [1.4143689 1.6642545] [1.4143689 1.6642545] [1.1718578 1.1757141] '''

tf.nn.sparse_softmax_cross_entropy_with_logits()

import tensorflow as tf ''' labels:labels[i]必须是[0, num_classes)的索引或者-1.如果是-1，则相应的损失为0,不用考虑 logits[i]的值。 logits:无尺度化的log概率 ''' labels = [0,1,9] #表示有三个样本的标签,第一个标签groundtruth是第0类,第二个标签groundtruth是第1类,,第三个标签groundtruth是第9类, logits = [[20.90,10.01,1.01,2.01,3.01,5.01,6.01,4.01,7.01,9.02,], # 预测值(注意不是概率)中第一个最大 [17.2,28.91,3.21,3.21,7.01,9.01,10.01,5.21,9.01,7.71], # 预测值(注意不是概率)中第二个最大 [10.0,0.0,40.0,50.0,0.0,0.0,0.0,0.0,0.0,100.0]] # 预测值(注意不是概率)中第十个最大 result = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=labels, logits=logits,) with tf.Session() as sess: print(sess.run(result)) ''' [2.6940936e-05 8.2254073e-06 0.0000000e+00] '''

tf.nn.sigmoid_cross_entropy_with_logits()

tf.nn.weighted_cross_entropy_with_logits()

tf.nn.sampled_softmax_loss()

tf.nn.scale_regularization_loss()

TensorFlow的tf.lossse模块里的损失函数计算方法（包括相关辅助函数）：

absolute_difference(…): Adds an Absolute Difference loss to the training procedure.

add_loss(…): Adds a externally defined loss to the collection of losses.

compute_weighted_loss(…): Computes the weighted loss.

cosine_distance(…): Adds a cosine-distance loss to the training procedure. (deprecated arguments)

get_losses(…): Gets the list of losses from the loss_collection.

get_regularization_loss(…): Gets the total regularization loss.

get_regularization_losses(…): Gets the list of regularization losses.

get_total_loss(…): Returns a tensor whose value represents the total loss.

hinge_loss(…): Adds a hinge loss to the training procedure.

huber_loss(…): Adds a Huber Loss term to the training procedure.

log_loss(…): Adds a Log Loss term to the training procedure.

mean_pairwise_squared_error(…): Adds a pairwise-errors-squared loss to the training procedure.

mean_squared_error(…): Adds a Sum-of-Squares loss to the training procedure.

sigmoid_cross_entropy(…): Creates a cross-entropy loss using tf.nn.sigmoid_cross_entropy_with_logits.

softmax_cross_entropy(…): Creates a cross-entropy loss using tf.nn.softmax_cross_entropy_with_logits_v2.

sparse_softmax_cross_entropy(…): Cross-entropy loss using tf.nn.sparse_softmax_cross_entropy_with_logits.