Theano tensor 模块：nnet 子模块

nnet 是 tensor 模块中与神经网络 Neural Networks 相关的子模块。

In [1]:

  1. import theano
  2. from theano import tensor as T
  1. Using gpu device 1: Tesla C2075 (CNMeM is disabled)

Sigmoid 函数

共有三种 sigmoid：

  • T.nnet.sigmoid(x)
  • T.nnet.ultra_sigmoid(x)
  • T.nnet.hard_sigmoid(x) 精度和时间：

sigmoid > ultra_fast_sigmoid > hard_sigmoid

函数图像：

09.18 Theano tensor 模块：nnet 子模块 - 图1

In [2]:

  1. x, y, b = T.dvectors('x', 'y', 'b')
  2. W = T.dmatrix('W')
  3. y = T.nnet.sigmoid(T.dot(W, x) + b)
  4.  
  5. print theano.pprint(y)
  1. sigmoid(((W \dot x) + b))

其他

T.nnet.softplus(x) 返回

\operatorname{softplus}(x) = \log_e{\left(1 + \exp(x)\right)}

会解决在 1 附近自定义函数值不准的问题。

In [3]:

  1. x,y,b = T.dvectors('x','y','b')
  2. W = T.dmatrix('W')
  3. y = T.nnet.softplus(T.dot(W,x) + b)
  4.  
  5. print theano.pprint(y)
  1. softplus(((W \dot x) + b))

T.nnet.softplus(x) 返回

\operatorname{softmax}{ij}(x) = \frac{\exp{x{ij}}}{\sumk\exp(x{ik})}

当 softmax 作用到矩阵时，它会按照行进行计算。

不过，下面的代码计算性能上更加稳定：

  1. e_x = exp(x - x.max(axis=1, keepdims=True))
  2. out = e_x / e_x.sum(axis=1, keepdims=True)

In [4]:

  1. x,y,b = T.dvectors('x','y','b')
  2. W = T.dmatrix('W')
  3. y = T.nnet.softmax(T.dot(W,x) + b)
  4.  
  5. print theano.pprint(y)
  1. Softmax(((W \dot x) + b))

T.nnet.relu(x, alpha=0) 返回这样一个函数：

f(x_i) = \left{\begin{aligned}x_i, & \ x_i > 0 \\alpha x_i, & \ otherwise\end{aligned}\right.

损失函数

T.nnet.binary_crossentropy(output, target) 二类交叉熵：

\text{crossentropy}(t,o) = -(t\cdot log(o) + (1 - t) \cdot log(1 - o))

In [5]:

  1. x, y, b, c = T.dvectors('x', 'y', 'b', 'c')
  2. W = T.dmatrix('W')
  3. V = T.dmatrix('V')
  4. h = T.nnet.sigmoid(T.dot(W, x) + b)
  5. x_recons = T.nnet.sigmoid(T.dot(V, h) + c)
  6. recon_cost = T.nnet.binary_crossentropy(x_recons, x).mean()

T.nnet.categorical_crossentropy(coding_dist, true_dist) 多类交叉熵

H(p,q) = - \sum_x p(x) \log(q(x))

原文: https://nbviewer.jupyter.org/github/lijin-THU/notes-python/blob/master/09-theano/09.18-tensor-nnet-.ipynb