2-1 Data Structure

Program = Data Structure + Algorithm

TensorFlow Program = Data Structure of Tensor + Algorithm in Graph

Tensor and graph are key concepts of TensorFlow.

The fundamental data structure in TensorFlow is Tensor, which is multi-dimentional array. Tensor is similar with the array in numpy.

There are two types of tensor accoring to the behavior: constant and variable.

The value of constant cannot be re-assigned in the graph, while variable can be re-assigned through operators such as assign.

1. Constant Tensor

The data type of tensor is basically corresponding to numpy.array.

  1. import numpy as np
  2. import tensorflow as tf
  4. i = tf.constant(1) # tf.int32 type constant
  5. l = tf.constant(1,dtype = tf.int64) # tf.int64 type constant
  6. f = tf.constant(1.23) #tf.float32 type constant
  7. d = tf.constant(3.14,dtype = tf.double) # tf.double type constant
  8. s = tf.constant("hello world") # tf.string type constant
  9. b = tf.constant(True) #tf.bool type constant
  12. print(tf.int64 == np.int64) 
  13. print(tf.bool == np.bool)
  14. print(tf.double == np.float64)
  15. print(tf.string == np.unicode) # tf.string type is not equal to np.unicode type
  1. True
  2. True
  3. True
  4. False

Each data type can be represented by tensor in different rank.

Scalars are tensors with rank = 0, arrays are with rank = 1, matrix are with rank = 2

Colorful image has three channels (RGB), which can be represented as a tensor with rank = 3.

There is a temporal dimension for video so it could be represented as a rank 4 tensor.

An intuitive way to understand: the number of the square brackets equals to the rank of the tensor.

  1. scalar = tf.constant(True)  #A scalar is a rank 0 tensor
  3. print(tf.rank(scalar))
  4. print(scalar.numpy().ndim)  # tf.rank equals to the ndim function in numpy
  1. tf.Tensor(0, shape=(), dtype=int32)
  2. 0
  1. vector = tf.constant([1.0,2.0,3.0,4.0]) #A vector is a rank 1 tensor
  3. print(tf.rank(vector))
  4. print(np.ndim(vector.numpy()))
  1. tf.Tensor(1, shape=(), dtype=int32)
  2. 1
  1. matrix = tf.constant([[1.0,2.0],[3.0,4.0]]) #A matrix is a rank 2 tensor
  3. print(tf.rank(matrix).numpy())
  4. print(np.ndim(matrix))
  1. 2
  2. 2
  1. tensor3 = tf.constant([[[1.0,2.0],[3.0,4.0]],[[5.0,6.0],[7.0,8.0]]])  # A rank 3 tensor
  2. print(tensor3)
  3. print(tf.rank(tensor3))
  1. tf.Tensor(
  2. [[[1. 2.]
  3.   [3. 4.]]
  5.  [[5. 6.]
  6.   [7. 8.]]], shape=(2, 2, 2), dtype=float32)
  7. tf.Tensor(3, shape=(), dtype=int32)
  1. tensor4 = tf.constant([[[[1.0,1.0],[2.0,2.0]],[[3.0,3.0],[4.0,4.0]]],
  2.                         [[[5.0,5.0],[6.0,6.0]],[[7.0,7.0],[8.0,8.0]]]])  # A rank 4 tensor
  3. print(tensor4)
  4. print(tf.rank(tensor4))
  1. tf.Tensor(
  2. [[[[1. 1.]
  3.    [2. 2.]]
  5.   [[3. 3.]
  6.    [4. 4.]]]
  9.  [[[5. 5.]
  10.    [6. 6.]]
  12.   [[7. 7.]
  13.    [8. 8.]]]], shape=(2, 2, 2, 2), dtype=float32)
  14. tf.Tensor(4, shape=(), dtype=int32)

We use tf.cast to change the data type of the tensors.

The method numpy() is for converting the data type from tensor to numpy array.

The method shape is for checking up the size of tensor.

  1. h = tf.constant([123,456],dtype = tf.int32)
  2. f = tf.cast(h,tf.float32)
  3. print(h.dtype, f.dtype)
  1. <dtype: 'int32'> <dtype: 'float32'>
  1. y = tf.constant([[1.0,2.0],[3.0,4.0]])
  2. print(y.numpy()) #Convert to np.array
  3. print(y.shape)
  1. [[1. 2.]
  2.  [3. 4.]]
  3. (2, 2)
  1. u = tf.constant(u"Hello World")
  2. print(u.numpy())  
  3. print(u.numpy().decode("utf-8"))
  1. b'\xe4\xbd\xa0\xe5\xa5\xbd \xe4\xb8\x96\xe7\x95\x8c'
  2. Hello World

2. Variable Tensor

The trainable parameters in the models are usually defined as variables.

  1. # The value of a constant is NOT changeable. Re-assignment creates a new space in the memory.
  2. c = tf.constant([1.0,2.0])
  3. print(c)
  4. print(id(c))
  5. c = c + tf.constant([1.0,1.0])
  6. print(c)
  7. print(id(c))
  1. tf.Tensor([1. 2.], shape=(2,), dtype=float32)
  2. 5276289568
  3. tf.Tensor([2. 3.], shape=(2,), dtype=float32)
  4. 5276290240
  1. # The value of a variable is changeable through re-assigning methods such as assign, assign_add, etc.
  2. v = tf.Variable([1.0,2.0],name = "v")
  3. print(v)
  4. print(id(v))
  5. v.assign_add([1.0,1.0])
  6. print(v)
  7. print(id(v))
  1. <tf.Variable 'v:0' shape=(2,) dtype=float32, numpy=array([1., 2.], dtype=float32)>
  2. 5276259888
  3. <tf.Variable 'v:0' shape=(2,) dtype=float32, numpy=array([2., 3.], dtype=float32)>
  4. 5276259888

Please leave comments in the WeChat official account “Python与算法之美” (Elegance of Python and Algorithms) if you want to communicate with the author about the content. The author will try best to reply given the limited time available.

You are also welcomed to join the group chat with the other readers through replying 加群 (join group) in the WeChat official account.

image.png