Making the double type

Type’s contract

In Theano’s framework, a Type (gof.type.Type)is any object which defines the followingmethods. To obtain the default methods described below, the Type shouldbe an instance of Type or should be an instance of asubclass of Type. If you will write all methods yourself,you need not use an instance of Type.

Methods with default arguments must be defined with the same signature,i.e. the same default argument names and values. If you wish to addextra arguments to any of these methods, these extra arguments must havedefault values.

• class PureType
• • filter(value, strict=False, allow_downcast=None)

  • This casts a value to match the Type and returns thecast value. If value is incompatible with the Type,the method must raise an exception. If strict is True, filter must return areference to value (i.e. casting prohibited).If strict is False, then casting may happen, but downcasting shouldonly be used in two situations:

    • if allow_downcast is True
    • if allow_downcast is None and the default behavior for thistype allows downcasting for the given value (this behavior istype-dependent, you may decide what your own type does by default) We need to define filter with three arguments. The second argumentmust be called strict (Theano often calls it by keyword) and musthave a default value of False. The third argument must be calledallow_downcast and must have a default value of None.

  • filterinplace(_value, storage, strict=False, allow_downcast=None)

  • If filter_inplace is defined, it will be called instead offilter() This is to allow reusing the old allocated memory. Asof this writing this is used only when we transfer new data to ashared variable on the gpu.

storage will be the old value. i.e. The old numpy array,CudaNdarray, …

• isvalid_value(_value)
• Returns True iff the value is compatible with the Type. Iffilter(value, strict = True) does not raise an exception, thevalue is compatible with the Type.

Default: True iff filter(value, strict=True) does not raisean exception.

• valueseq(_a, b)
• Returns True iff a and b are equal.

Default: a == b

• valueseq_approx(_a, b)
• Returns True iff a and b are approximately equal, for adefinition of “approximately” which varies from Type to Type.

Default: values_eq(a, b)

• makevariable(_name=None)
• Makes a Variable of this Type with the specified name, ifname is not None. If name is None, then the Variable doesnot have a name. The Variable will have its type field set tothe Type object.

Default: there is a generic definition of this in Type. TheVariable’s type will be the object that defines this method (inother words, self).

• call(name=None)
• Syntactic shortcut to make_variable.

Default: make_variable

• eq(other)
• Used to compare Type instances themselves

Default: object.eq

• hash()
• Types should not be mutable, so it should be OK to define a hashfunction. Typically this function should hash all of the termsinvolved in eq.

Default: id(self)

• getshape_info(_obj)
• Optional. Only needed to profile the memory of this Type of object.

Return the information needed to compute the memory size of obj.

The memory size is only the data, so this excludes the container.For an ndarray, this is the data, but not the ndarray object andother data structures such as shape and strides.

get_shape_info() and get_size() work in tandem for the memory profiler.

get_shape_info() is called during the execution of the function.So it is better that it is not too slow.

get_size() will be called on the output of this functionwhen printing the memory profile.

Parameters:obj – The object that this Type represents during executionReturns:Python object that self.get_size() understands

• getsize(_shape_info)
• Number of bytes taken by the object represented by shape_info.

Optional. Only needed to profile the memory of this Type of object.

Parameters:shape_info – the output of the call to get_shape_info()Returns:the number of bytes taken by the object described byshape_info.

• clone(dtype=None, broadcastable=None)
• Optional, for TensorType-alikes.

Return a copy of the type with a possibly changed value fordtype and broadcastable (if they aren’t None).

Parameters:

- **dtype** – New dtype for the copy.
- **broadcastable** – New broadcastable tuple for the copy.

• mayshare_memory(_a, b)
• Optional to run, but mandatory for DebugMode. Return True if the Pythonobjects a and b could share memory. Return Falseotherwise. It is used to debug when Ops did not declare memoryaliasing between variables. Can be a static method.It is highly recommended to use and is mandatory for Type in Theanoas our buildbot runs in DebugMode.

For each method, the default is what Type definesfor you. So, if you create an instance of Type or aninstance of a subclass of Type, youmust define filter. You might want to override values_eq_approx,as well as values_eq. The other defaults generally need not beoverridden.

For more details you can go see the documentation for Type.

Additional definitions

For certain mechanisms, you can register functions and other suchthings to plus your type into theano’s mechanisms. These are optionalbut will allow people to use you type with familiar interfaces.

transfer()

To plug in additional options for the transfer target, define afunction which takes a theano variable and a target argument andreturns eitehr a new transferred variable (which can be the same asthe input if no transfer is nessecary) or returns None if the transfercan’t be done.

Then register that function by calling register_transfer()with it as argument.

Defining double

We are going to base Type double on Python’s float. Wemust define filter and shall override values_eq_approx.

filter

# Note that we shadow Python's function ``filter`` with this
# definition.
def filter(x, strict=False, allow_downcast=None):
    if strict:
        if isinstance(x, float):
            return x
        else:
            raise TypeError('Expected a float!')
    elif allow_downcast:
        return float(x)
    else:   # Covers both the False and None cases.
        x_float = float(x)
        if x_float == x:
            return x_float
        else:
             raise TypeError('The double type cannot accurately represent '
                             'value %s (of type %s): you must explicitly '
                             'allow downcasting if you want to do this.'
                             % (x, type(x)))

If strict is True we need to return x. If strict is True and x is not afloat (for example, x could easily be an int) then it isincompatible with our Type and we must raise an exception.

If strict is False then we are allowed to cast x to a float,so if x is an int it we will return an equivalent float.However if this cast triggers a precision loss (x != float(x)) andallow_downcast is not True, then we also raise an exception.Note that here we decided that the default behavior of our type(when allow_downcast is set to None) would be the same aswhen allow_downcast is False, i.e. no precision loss is allowed.

values_eq_approx

def values_eq_approx(x, y, tolerance=1e-4):
    return abs(x - y) / (abs(x) + abs(y)) < tolerance

The second method we define is values_eq_approx. This methodallows approximate comparison between two values respecting our Type’sconstraints. It might happen that an optimization changes the computationgraph in such a way that it produces slightly different variables, forexample because of numerical instability like rounding errors at theend of the mantissa. For instance, a + a + a + a + a + a might notactually produce the exact same output as 6 * a (try with a=0.1),but with values_eq_approx we do not necessarily mind.

We added an extra tolerance argument here. Since this argument isnot part of the API, it must have a default value, which wechose to be 1e-4.

Note

values_eq is never actually used by Theano, but it might be usedinternally in the future. Equality testing inDebugMode is done using values_eq_approx.

Putting them together

What we want is an object that respects the aforementionedcontract. Recall that Type defines default implementations for allrequired methods of the interface, except filter. One way to makethe Type is to instantiate a plain Type and set the needed fields:

from theano import gof
 
double = gof.Type()
double.filter = filter
double.values_eq_approx = values_eq_approx

Another way to make this Type is to make a subclass of gof.Typeand define filter and values_eq_approx in the subclass:

from theano import gof
 
class Double(gof.Type):
 
    def filter(self, x, strict=False, allow_downcast=None):
        # See code above.
        ...
 
    def values_eq_approx(self, x, y, tolerance=1e-4):
        # See code above.
        ...
 
double = Double()

double is then an instance of Type Double, which in turn is asubclass of Type.

There is a small issue with defining double this way. Allinstances of Double are technically the same Type. However, differentDouble Type instances do not compare the same:

>>> double1 = Double()
>>> double2 = Double()
>>> double1 == double2
False

Theano compares Types using == to see if they are the same.This happens in DebugMode. Also, Ops can (and should) ensure that their inputshave the expected Type by checking something like if x.type == lvector.

There are several ways to make sure that equality testing works properly:

1. Define Double.eq so that instances of type Double are equal. For example:

   def eq(self, other):
       return type(self) is Double and type(other) is Double

2. Override Double.new to always return the same instance.

3. Hide the Double class and only advertise a single instance of it.

Here we will prefer the final option, because it is the simplest.Ops in the Theano code often define the eq method though.

Untangling some concepts

Initially, confusion is common on what an instance of Type is versusa subclass of Type or an instance of Variable. Some of this confusion issyntactic. A Type is any object which has fields corresponding to thefunctions defined above. The Type class provides sensible defaults forall of them except filter, so when defining new Types it is naturalto subclass Type. Therefore, we often end up with Type subclasses andit is can be confusing what these represent semantically. Here is anattempt to clear up the confusion:

• An instance of Type (or an instance of a subclass)is a set of constraints on real data. It isakin to a primitive type or class in C. It is a _static_annotation.
• An instance of Variable symbolizes data nodes in a data flowgraph. If you were to parse the C expression int x;, intwould be a Type instance and x would be a Variable instance ofthat Type instance. If you were to parse the C expression c = a + b;, a, b and c would all be Variable instances.
• A subclass of Type is a way of implementinga set of Type instances that sharestructural similarities. In the double example that we are doing,there is actually only one Type in that set, therefore the subclassdoes not represent anything that one of its instances does not. In thiscase it is a singleton, a set with one element. However, theTensorTypeclass in Theano (which is a subclass of Type)represents a set of types of tensorsparametrized by their data type or number of dimensions. We could saythat subclassing Type builds a hierarchy of Types which is based uponstructural similarity rather than compatibility.

Final version

from theano import gof
 
class Double(gof.Type):
 
    def filter(self, x, strict=False, allow_downcast=None):
        if strict:
            if isinstance(x, float):
                return x
            else:
                raise TypeError('Expected a float!')
        elif allow_downcast:
            return float(x)
        else:   # Covers both the False and None cases.
            x_float = float(x)
            if x_float == x:
                return x_float
            else:
                 raise TypeError('The double type cannot accurately represent '
                                 'value %s (of type %s): you must explicitly '
                                 'allow downcasting if you want to do this.'
                                 % (x, type(x)))
 
    def values_eq_approx(self, x, y, tolerance=1e-4):
        return abs(x - y) / (abs(x) + abs(y)) < tolerance
 
    def __str__(self):
        return "double"
 
double = Double()

We add one utility function, str. That way, when we printdouble, it will print out something intelligible.