Tensorflow slim模块

最近在看tf-faster-rcnn的代码,看到了VGG16网络的定义中使用到了slim这个模块,如下图所示,因此百度了一下slim这个模块,下面的内容是基于一些博客按照自己的思路整理的,便于理解和日后查用。

《Tensorflow slim模块》 VGG16.jpeg

slim这个模块是在16年新推出的,主要是来做所谓的“代码瘦身”, 可以消除原生tensorflow里面很多重复的模板性的代码,让代码更紧凑,更具备可读性。另外slim提供了很多计算机视觉方面的著名模型(VGG, ResNet等),我们可以直接下载使用(checkpoint,即 .ckpt文件)。

《Tensorflow slim模块》 Pre-trained Models.jpeg slim被放在tensorflow.contrib这个库下面,导入的方法如下:

import tensorflow.contrib.slim as slim

下面分别介绍slim的代码瘦身功能和slim中一些常用函数。

一、代码瘦身功能:

1、首先让我们看看tensorflow怎么实现一个层,例如卷积层:

input = ...
with tf.name_scope('conv1_1') as scope:
  kernel = tf.Variable(tf.truncated_normal([3, 3, 64, 128], dtype=tf.float32,
                                           stddev=1e-1), name='weights')
  conv = tf.nn.conv2d(input, kernel, [1, 1, 1, 1], padding='SAME')
  biases = tf.Variable(tf.constant(0.0, shape=[128], dtype=tf.float32),
                       trainable=True, name='biases')
  bias = tf.nn.bias_add(conv, biases)
  conv1 = tf.nn.relu(bias, name=scope)

然后slim的实现:

input = ...
net = slim.conv2d(input, 128, [3, 3], scope='conv1_1')

2、repeat操作
假设在tensorflow中定义三个相同的卷积层:

net = ...
net = slim.conv2d(net, 256, [3, 3], scope='conv3_1')
net = slim.conv2d(net, 256, [3, 3], scope='conv3_2')
net = slim.conv2d(net, 256, [3, 3], scope='conv3_3')
net = slim.max_pool2d(net, [2, 2], scope='pool2')

在slim中的repeat操作:

net = slim.repeat(net, 3, slim.conv2d, 256, [3, 3], scope='conv3')
net = slim.max_pool2d(net, [2, 2], scope='pool2')

3、stack操作:处理卷积核或者输出不一样的情况
假设在tensorflow中定义三层FC:

x = slim.fully_connected(x, 32, scope='fc/fc_1')
x = slim.fully_connected(x, 64, scope='fc/fc_2')
x = slim.fully_connected(x, 128, scope='fc/fc_3')

在slim中的stack操作:

slim.stack(x, slim.fully_connected, [32, 64, 128], scope='fc')

同理卷积层也一样:

# 普通方法:
x = slim.conv2d(x, 32, [3, 3], scope='core/core_1')
x = slim.conv2d(x, 32, [1, 1], scope='core/core_2')
x = slim.conv2d(x, 64, [3, 3], scope='core/core_3')
x = slim.conv2d(x, 64, [1, 1], scope='core/core_4')
 
# 简便方法:
slim.stack(x, slim.conv2d, [(32, [3, 3]), (32, [1, 1]), (64, [3, 3]), (64, [1, 1])], scope='core')

4、slim中的argscope:如果网络有大量相同的参数,如下:

net = slim.conv2d(inputs, 64, [11, 11], 4, padding='SAME',
                  weights_initializer=tf.truncated_normal_initializer(stddev=0.01),
                  weights_regularizer=slim.l2_regularizer(0.0005), scope='conv1')
net = slim.conv2d(net, 128, [11, 11], padding='VALID',
                  weights_initializer=tf.truncated_normal_initializer(stddev=0.01),
                  weights_regularizer=slim.l2_regularizer(0.0005), scope='conv2')
net = slim.conv2d(net, 256, [11, 11], padding='SAME',
                  weights_initializer=tf.truncated_normal_initializer(stddev=0.01),
                  weights_regularizer=slim.l2_regularizer(0.0005), scope='conv3')

然后用arg_scope处理一下:

with slim.arg_scope([slim.conv2d], padding='SAME',
                      weights_initializer=tf.truncated_normal_initializer(stddev=0.01)
                      weights_regularizer=slim.l2_regularizer(0.0005)):
    net = slim.conv2d(inputs, 64, [11, 11], scope='conv1')
    net = slim.conv2d(net, 128, [11, 11], padding='VALID', scope='conv2')
    net = slim.conv2d(net, 256, [11, 11], scope='conv3')

Tips:arg_scope的作用范围内,定义了指定层的默认参数,若想特别指定某些层的参数,可以重新赋值(相当于重写),如上倒数第二行代码。那如果除了卷积层还有其他层呢?那就要如下定义:

with slim.arg_scope([slim.conv2d, slim.fully_connected],
                      activation_fn=tf.nn.relu,
                      weights_initializer=tf.truncated_normal_initializer(stddev=0.01),
                      weights_regularizer=slim.l2_regularizer(0.0005)):
  with slim.arg_scope([slim.conv2d], stride=1, padding='SAME'):
    net = slim.conv2d(inputs, 64, [11, 11], 4, padding='VALID', scope='conv1')
    net = slim.conv2d(net, 256, [5, 5],
                      weights_initializer=tf.truncated_normal_initializer(stddev=0.03),
                      scope='conv2')
    net = slim.fully_connected(net, 1000, activation_fn=None, scope=

写两个arg_scope就行了。

基于以上的“瘦身”后,定义一个VGG16,二十来行代码就搞定了。

def vgg16(inputs):
  with slim.arg_scope([slim.conv2d, slim.fully_connected],
                      activation_fn=tf.nn.relu,
                      weights_initializer=tf.truncated_normal_initializer(0.0, 0.01),
                      weights_regularizer=slim.l2_regularizer(0.0005)):
    net = slim.repeat(inputs, 2, slim.conv2d, 64, [3, 3], scope='conv1')
    net = slim.max_pool2d(net, [2, 2], scope='pool1')
    net = slim.repeat(net, 2, slim.conv2d, 128, [3, 3], scope='conv2')
    net = slim.max_pool2d(net, [2, 2], scope='pool2')
    net = slim.repeat(net, 3, slim.conv2d, 256, [3, 3], scope='conv3')
    net = slim.max_pool2d(net, [2, 2], scope='pool3')
    net = slim.repeat(net, 3, slim.conv2d, 512, [3, 3], scope='conv4')
    net = slim.max_pool2d(net, [2, 2], scope='pool4')
    net = slim.repeat(net, 3, slim.conv2d, 512, [3, 3], scope='conv5')
    net = slim.max_pool2d(net, [2, 2], scope='pool5')
    net = slim.fully_connected(net, 4096, scope='fc6')
    net = slim.dropout(net, 0.5, scope='dropout6')
    net = slim.fully_connected(net, 4096, scope='fc7')
    net = slim.dropout(net, 0.5, scope='dropout7')
    net = slim.fully_connected(net, 1000, activation_fn=None, scope='fc8')
  return net

二、常用函数:
1、slim.arg_scope: slim.arg_scope: 可以定义一些函数的默认参数值,在scope内,我们重复用到这些函数时可以不用把所有参数都写一遍。

with slim.arg_scope([slim.conv2d, slim.fully_connected], 
                    trainable=True,
                    activation_fn=tf.nn.relu, 
                    weights_initializer=tf.truncated_normal_initializer(stddev=0.01), 
                    weights_regularizer=slim.l2_regularizer(0.0001)):
    with slim.arg_scope([slim.conv2d], 
                        kernel_size=[3, 3], 
                        padding='SAME',
                        normalizer_fn=slim.batch_norm):
        net = slim.conv2d(net, 64, scope='conv1'))
        net = slim.conv2d(net, 128, scope='conv2'))
        net = slim.conv2d(net, 256, [5, 5], scope='conv3'))

slim.arg_scope的用法基本都体现在上面了。一个slim.arg_scope内可以用list来同时定义多个函数的默认参数(前提是这些函数都有这些参数),另外,slim.arg_scope也允许相互嵌套。在其中调用的函数,可以不用重复写一些参数(例如kernel_size=[3, 3]),但也允许覆盖(例如最后一行,卷积核大小为[5,5])。
2、slim.cov2d: 卷积层,一般调用方法如下:

net = slim.conv2d(inputs, 256, [3, 3], stride=1, scope='conv1_1')

前三个参数依次为网络的输入,输出的通道,卷积核大小,stride是做卷积时的步长。除此之外,还有几个经常被用到的参数:

padding : 补零的方式,例如'SAME'
activation_fn : 激活函数,默认是nn.relu, VGG16中就用的是ReLU
normalizer_fn : 正则化函数,默认为None,这里可以设置为batch normalization,函数用slim.batch_norm
normalizer_params : slim.batch_norm中的参数,以字典形式表示
weights_initializer : 权重的初始化器,initializers.xavier_initializer()
weights_regularizer : 权重的正则化器,一般不怎么用到
biases_initializer : 如果之前有batch norm,那么这个及下面一个就不用管了
biases_regularizer : 
trainable : 参数是否可训练,默认为True

3、slim.max_pool2d: 池化层(最大池化和平均池化),用法如下:

net = slim.max_pool2d(net, [2, 2], scope='pool1')

4、slim.fully_connected: 全连接层,前两个参数分别为网络输入、输出的神经元数量。

slim.fully_connected(x, 128, scope='fc1')
    原文作者:素娜93
    原文地址: https://www.jianshu.com/p/e2ada4ddae9a
    本文转自网络文章,转载此文章仅为分享知识,如有侵权,请联系博主进行删除。
点赞