1 跟踪训练函数的损失值

如果跑的代码中没有显示损失函数的变化趋势，而自己需要根据这个来调整超参，那么可以自己编写函数去实现这一个需求。 首先应该将每次的损失值记录并储存下来。这里以CornerNet的代码为例，代码链接如下： CornerNet 在train.py函数中，损失值在如下的部分得到：

with stdout_to_tqdm() as save_stdout: for iteration in tqdm(range(start_iter + 1, max_iteration + 1), file=save_stdout, ncols=80): training = pinned_training_queue.get(block=True) training_loss = nnet.train(**training) if display and iteration % display == 0: print("training loss at iteration {}: {}".format(iteration, training_loss.item())) del training_loss # if val_iter and validation_db.db_inds.size and iteration % val_iter == 0: # nnet.eval_mode() # validation = pinned_validation_queue.get(block=True) # validation_loss = nnet.validate(**validation) # print("validation loss at iteration {}: {}".format(iteration, validation_loss.item())) # nnet.train_mode() if iteration % snapshot == 0: nnet.save_params(iteration) if iteration % stepsize == 0: learning_rate /= decay_rate nnet.set_lr(learning_rate)

如果想要记录并储存好其值，加入以下代码即可：

with stdout_to_tqdm() as save_stdout: for iteration in tqdm(range(start_iter + 1, max_iteration + 1), file=save_stdout, ncols=80): training = pinned_training_queue.get(block=True) training_loss = nnet.train(**training) loss = training_loss.cpu() loss_ = str(loss.data.numpy()) with open('path', 'a') as f: f.write(str(iteration)) f.write(' ') f.write(loss_) if iteration < max_iteration: f.write(' \r\n') if display and iteration % display == 0: print("training loss at iteration {}: {}".format(iteration, training_loss.item())) del training_loss # if val_iter and validation_db.db_inds.size and iteration % val_iter == 0: # nnet.eval_mode() # validation = pinned_validation_queue.get(block=True) # validation_loss = nnet.validate(**validation) # print("validation loss at iteration {}: {}".format(iteration, validation_loss.item())) # nnet.train_mode() if iteration % snapshot == 0: nnet.save_params(iteration) if iteration % stepsize == 0: learning_rate /= decay_rate nnet.set_lr(learning_rate)

加入的部分为：

loss = training_loss.cpu() loss_ = str(loss.data.numpy()) with open('path', 'a') as f: f.write(str(iteration)) f.write(' ') f.write(loss_) if iteration < max_iteration: f.write(' \r\n')

解释一下代码 由于深度学习计算loss的时候基本上loss都是cuda的一个tensor变量，储存在cuda中的。是不能够直接复制过来的，所以需要用.cpu()把cuda中的值转移给cpu中储存 然后由于此时转移过去以后还是一个variable变量（就是可以backpropogation来计算grad的一种变量），所以需要用variable.data把其中的数据单独取出来，但是此时还是一个tensor，需要转换成numpy，所以再通过一个.numpy() f.write（）中的必须是一个string类型的数据，所以要转成string 最后if语句是为了在最后一行的时候不要再写入回车了。 path:就是你想把记录下来的数据储存在哪个文件夹下，比如./loss.txt，就是储存在当前路径下的txt文件。这里建议保存成txt文件，以后要用的话就很方便处理。这个txt文件不需要自己新建一个空白，没有的话python会自己建立一个的 open函数中我选择用a参数，而不是用w，这是因为w会抹去之前写过的重新写，最后只剩下最后一对数据了，而我们的目的明显不是这样的，a是从上一次的位置接着写，这就nice了。

2 根据保存好的训练过程中的损失值可视化

""" Note: The code is used to show the change trende via the whole training procession. First: You need to mark all the loss of every iteration Second: You need to write these data into a txt file with the format like: ...... iter loss iter loss ...... Third: the path is the txt file path of your loss """ import matplotlib.pyplot as plt def read_txt(path): with open(path, 'r') as f: lines = f.readlines() splitlines = [x.strip().split(' ') for x in lines] return splitlines # Referenced from Tensorboard(a smooth_loss function:https://blog.csdn.net/charel_chen/article/details/80364841) def smooth_loss(path, weight=0.85): iter = [] loss = [] data = read_txt(path) for value in data: iter.append(int(value[0])) loss.append(int(float(value[1]))) # Note a str like '3.552' can not be changed to int type directly # You need to change it to float first, can then you can change the float type ton int type last = loss[0] smoothed = [] for point in loss: smoothed_val = last * weight + (1 - weight) * point smoothed.append(smoothed_val) last = smoothed_val return iter, smoothed if __name__ == "__main__": path = './loss.txt' loss = [] iter = [] iter, loss = smooth_loss(path) plt.plot(iter, loss, linewidth=2) plt.title("Loss-iters", fontsize=24) plt.xlabel("iters", fontsize=14) plt.ylabel("loss", fontsize=14) plt.tick_params(axis='both', labelsize=14) plt.savefig('./loss_func.png') plt.show()

这里借用了tensorboard 平滑损失曲线代码