函数的解析

<div class="blogpost-body" id="cnblogs_post_body">
tensorflow 函数解析
1.optimizer.minimize(loss, var_list)
TensorFlow为我们提供了丰富的优化函数，例如GradientDescentOptimizer。这个方法会自动根据loss计算对应variable的导数。示例如下：
<div class="cnblogs_Highlighter">
 <pre class="blockcode"><code class="language-python">loss = ...
opt = tf.tf.train.GradientDescentOptimizer(learning_rate=0.1)
train_op = opt.minimize(loss)
init = tf.initialize_all_variables()

with tf.Seesion() as sess:
sess.run(init)
for step in range(10):
 session.run(train_op)
</code></pre>
</div>
　看一下<code>minimize()</code>的源代码(为方便说明，部分参数已删除):
<div class="cnblogs_code">
 <pre class="blockcode"> 1 def minimize(self, loss, global_step=None, var_list=None, name=None):
 2
 3 grads_and_vars = self.compute_gradients(loss, var_list=var_list)
 4
 5 vars_with_grad = [v for g, v in grads_and_vars if g is not None]
 6 if not vars_with_grad:
 7 raise ValueError(
 8 "No gradients provided for any variable, check your graph for ops"
 9 " that do not support gradients, between variables %s and loss %s." %
10 ([str(v) for _, v in grads_and_vars], loss))
11
12 return self.apply_gradients(grads_and_vars, global_step=global_step,
13 name=name)</pre>
</div>
源代码可以知道<code>minimize()</code>实际上包含了两个步骤，即<code>compute_gradients</code>和<code>apply_gradients</code>，前者用于计算梯度，后者用于使用计算得到的梯度来更新对应的variable，<a href="https://blog.csdn.net/NockinOnHeavensDoor/article/details/80632677#%E6%A2%AF%E5%BA%A6%E4%BF%AE%E5%89%AA%E4%B8%BB%E8%A6%81%E9%81%BF%E5%85%8D%E8%AE%AD%E7%BB%83%E6%A2%AF%E5%BA%A6%E7%88%86%E7%82%B8%E5%92%8C%E6%B6%88%E5%A4%B1%E9%97%AE%E9%A2%98">梯度修剪主要避免训练梯度爆炸和消失问题</a>。下面对这两个函数做具体介绍。
1.1 computer_gradients(loss, val_list)
参数含义:
<ul><li>loss: 需要被优化的Tensor</li><li>val_list: Optional list or tuple of <code>tf.Variable</code> to update to minimize <code>loss</code>. Defaults to the list of variables collected in the graph under the key <code>GraphKeys.TRAINABLE_VARIABLES</code>.</li></ul>
简单说该函数就是用于计算loss对于指定val_list的导数的，最终返回的是元组列表，即[(gradient, variable),...]。
<div class="cnblogs_code">
 <pre class="blockcode">1 x = tf.Variable(initial_value=50., dtype='float32')
2 w = tf.Variable(initial_value=10., dtype='float32')
3 y = w*x
4
5 opt = tf.train.GradientDescentOptimizer(0.1)
6 grad = opt.compute_gradients(y, [w,x])
7 with tf.Session() as sess:
8 sess.run(tf.global_variables_initializer())
9 print(sess.run(grad))</pre>
</div>
<div class="cnblogs_code">
 <pre class="blockcode">>>> [(50.0, 10.