nanguardmode

Guide

The NanGuardMode aims to prevent the model from outputting NaNs or Infs. It hasa number of self-checks, which can help to find out which apply node isgenerating those incorrect outputs. It provides automatic detection of 3 typesof abnormal values: NaNs, Infs, and abnormally big values.

NanGuardMode can be used as follows:

  1. import numpy
  2. import theano
  3. import theano.tensor as T
  4. from theano.compile.nanguardmode import NanGuardMode
  5.  
  6. x = T.matrix()
  7. w = theano.shared(numpy.random.randn(5, 7).astype(theano.config.floatX))
  8. y = T.dot(x, w)
  9. fun = theano.function(
  10.     [x], y,
  11.     mode=NanGuardMode(nan_is_error=True, inf_is_error=True, big_is_error=True)
  12. )

While using the theano function fun, it will monitor the values of eachinput and output variable of each node. When abnormal values aredetected, it raises an error to indicate which node yields the NaNs. Forexample, if we pass the following values to fun:

  1. infa = numpy.tile(
  2.     (numpy.asarray(100.) ** 1000000).astype(theano.config.floatX), (3, 5))
  3. fun(infa)

It will raise an AssertionError indicating that Inf value is detected whileexecuting the function.

You can also set the three parameters in NanGuardMode() to indicate whichkind of abnormal values to monitor. nan_is_error and inf_is_error hasno default values, so they need to be set explicitly, but big_is_error isset to be True by default.

Note

NanGuardMode significantly slows down computations; onlyenable as needed.

Reference

  • class theano.compile.nanguardmode.NanGuardMode(nan_is_error=None, inf_is_error=None, big_is_error=None, optimizer='default', linker=None)[source]
  • A Theano compilation Mode that makes the compiled function automaticallydetect NaNs and Infs and detect an error if they occur.

Parameters:

  • nan_is_error (bool) – If True, raise an error anytime a NaN is encountered.
  • inf_is_error (bool) – If True, raise an error anytime an Inf is encountered. Note that somepylearn2 modules currently use np.inf as a default value (e.g.mlp.max_pool) and these will cause an error if inf_is_error is True.
  • big_is_error (bool) – If True, raise an error when a value greater than 1e10 is encountered.

Note

We ignore the linker parameter