TENSORS

The data in the neural network is stored in tensors, which are similar to arrays and mathematical matrices. OneFlow provides a series of operators on tensors. Tensors, together with operators, build up a neural network.

Tensors differ from regular multidimensional arrays in that they can run on AI chips (such as the Nvidia GPU) and CPU, thus increasing computing speed. In addition, OneFlow provides Autograd function, which supports automatic differentiation.

  1. import oneflow as flow
  2. import numpy as np

Creating Tensors

There are several ways to create tensors, including:

  • Directly from data
  • From a NumPy array
  • By an operator

Directly from data

Tensors can be created directly from data:

  1. x1 = flow.tensor([[1, 2], [3, 4]])
  2. x2 = flow.tensor([[1.0, 2.0], [3.0, 4.0]])
  3. print(x1)
  4. print(x2)

Out:

  1. tensor([[1, 2],
  2.         [3, 4]], dtype=oneflow.int64)
  3. tensor([[1., 2.],
  4.         [3., 4.]], dtype=oneflow.float32)

We can see that the tensor x1 and x2 are created, whose data types are int64 and float32, respectively.

From a NumPy array

Tensors can be created from NumPy arrays by passing the NumPy array as a parameter when the tensor object is constructed.

  1. x3 = flow.tensor(np.ones((2,3)))
  2. x4 = flow.tensor(np.random.rand(2,3))
  3. print(x3)
  4. print(x4)

Out:

  1. tensor([[1., 1., 1.],
  2.         [1., 1., 1.]], dtype=oneflow.float64)
  3. tensor([[0.6213, 0.6142, 0.1592],
  4.         [0.5539, 0.8453, 0.8576]], dtype=oneflow.float64)

By an operator

There are also many operators available in OneFlow that can be used to create tensors. For example, ones, zeros and eye, which create the all-ones tensor, zero tensor, and identity tensor, respectively.

  1. x5 = flow.ones(2, 3)
  2. x6 = flow.zeros(2, 3)
  3. x7 = flow.eye(3)
  4. print(x5)
  5. print(x6)
  6. print(x7)

Out:

  1. tensor([[1., 1., 1.],
  2.         [1., 1., 1.]], dtype=oneflow.float32)
  3. tensor([[0., 0., 0.],
  4.         [0., 0., 0.]], dtype=oneflow.float32)
  5. tensor([[1., 0., 0.],
  6.         [0., 1., 0.],
  7.         [0., 0., 1.]], dtype=oneflow.float32)

The randn method creates a random tensor:

  1. x8 = flow.randn(2,3)

Difference Between Tensor and tensor

There are two interfaces (oneflow.Tensor and oneflow.tensor) in OneFlow, both of which can be used to create tensors. What’s the difference between them?

Briefly speaking, the data type of oneflow.Tensor is limited to float32 by default, while the data type of oneflow.tensor can be changed when the data is created. The following code illustrates the difference:

  1. print(flow.Tensor([1, 2, 3]))
  2. print(flow.tensor([1, 2, 3]))
  3. print(flow.tensor([1.0, 2.0, 3.0]))

Out:

  1. tensor([1., 2., 3.], dtype=oneflow.float32)
  2. tensor([1, 2, 3], dtype=oneflow.int64)
  3. tensor([1., 2., 3.], dtype=oneflow.float32)

Besides, oneflow.Tensor can be created without specific data:

  1. x9 = flow.Tensor(2, 3)
  2. print(x9.shape)

Out:

  1. flow.Size([2, 3])

Therefore, use oneflow.Tensor to create a tensor if you do not want to specify an explicit value, otherwise, you should use oneflow.tensor.

Attributes of a Tensor

The shape, dtype, and device attributes of a tensor describe its shape, data type, and device type respectively.

  1. x9 = flow.randn(1,4)
  2. print(x9.shape)
  3. print(x9.dtype)
  4. print(x9.device)

Out:

  1. flow.Size([1, 4])
  2. oneflow.float32
  3. cpu:0

The output shows the shape, the data type, and the device (on CPU No. 0, CPUs were numbered because OneFlow naturally supports distribution, see Consistent Tensor).

The shape of the tensor can be changed by the reshape method, and the data type and device of the tensor can be changed by the to method:

  1. x10 = x9.reshape(2, 2)
  2. x11 = x10.to(dtype=flow.int32, device=flow.device("cuda"))
  3. print(x10.shape)
  4. print(x11.dtype, x11.device)

Out:

  1. flow.Size([2, 2])
  2. oneflow.int32 cuda:0

Operations on Tensors

A large number of operators are provided in OneFlow, most of which are in the namespaces of oneflow, oneflow.nn, and oneflow.nn.functional.

Tensors in OneFlow are as easy to use as the NumPy arrays. For example, slicing in NumPy style is supported:

  1. tensor = flow.ones(4, 4)
  2. print('First row: ',tensor[0])
  3. print('First column: ', tensor[:, 0])
  4. print('Last column:', tensor[..., -1])
  5. tensor[:,1] = 0
  6. print(tensor)

Out:

  1. First row:  tensor([1., 1., 1., 1.], dtype=oneflow.float32)
  2. First column:  tensor([1., 1., 1., 1.], dtype=oneflow.float32)
  3. Last column: tensor([1., 1., 1., 1.], dtype=oneflow.float32)
  4. tensor([[1., 0., 1., 1.],
  5.         [1., 0., 1., 1.],
  6.         [1., 0., 1., 1.],
  7.         [1., 0., 1., 1.]], dtype=oneflow.float32)

In addition, there are many other operations in OneFlow, such as add, sub, mul, div for arithmetic operations; scatter, gather, gather_nd for positional operations; and activation functions (relu), convolution functions (conv2d), etc. Click on their links to see detailed API description and find out more about other operators.

Please activate JavaScript for write a comment in LiveRe