Pytorch Indexbased Operation (original) (raw)

PyTorch is a python library developed by Facebook to run and train deep learning and machine learning algorithms. Tensor is the fundamental data structure of the machine or deep learning algorithms and to deal with them, we perform several operations, for which PyTorch library offers many functionalities.

Tensor operations that handle indexing on some particular row or column for copying, adding, filling values/tensors are said to be index-based developed operation. There are two types of index-based operations in PyTorch, one is in-place operations and the other is out-of-place operations. The basic difference between the two is in-place operation directly changes the values of the tensors without making any copy of that whereas out of place operations don’t. Following are the operations:-

  1. index_add_
  2. index_add
  3. index_copy_
  4. index_copy
  5. index_fill_
  6. index_fill
  7. index_put_
  8. index_put
  9. index_select

Now we discuss each of the functions with proper examples.

1. index_add_: Adds the given tensor elements to the self tensor along the order given in the matrix.

Syntax:

index_add_(dim,index,ensor)---> Tensor

Parameters:

Example 1: We take a zero vector ‘x’, te tensor of size(3,5) and index tensor. Accumulating the resultant vector along rows we get the output.

Python3

import torch

x = torch.zeros( 5 , 5 )

te = torch.tensor([[ 1 , 3 , 5 , 7 , 9 ],[ 1 , 3 , 5 , 7 , 9 ],[ 1 , 3 , 5 , 7 , 9 ]],dtype = torch.float32)

index0 = torch.tensor([ 0 , 2 , 4 ])

x.index_add_( 0 ,index0,te)

Output:

tensor([[1., 3., 5., 7., 9.], [0., 0., 0., 0., 0.], [1., 3., 5., 7., 9.], [0., 0., 0., 0., 0.], [1., 3., 5., 7., 9.]])

Example 2:

Python3

import torch

y = torch.ones( 5 , 5 )

index2 = torch.tensor([ 0 , 1 , 1 , 1 , 2 ])

ten = torch.randn( 1 , 5 )

y.index_add_( 1 ,index2,ten)

Output:

tensor([[0.9460, 0.4762, 1.2219, 1.0000, 1.0000], [0.9460, 0.4762, 1.2219, 1.0000, 1.0000], [0.9460, 0.4762, 1.2219, 1.0000, 1.0000], [0.9460, 0.4762, 1.2219, 1.0000, 1.0000], [0.9460, 0.4762, 1.2219, 1.0000, 1.0000]])

2.index_add: It is the out-of place version of the above function. This adds a given tensor to the self tensor temporarily. Parameters and syntax are the out-of-placesame as above.

Example 3:

Python3

import torch

y = torch.ones( 5 , 5 )

index2 = torch.tensor([ 0 , 1 , 1 , 1 , 2 ])

ten = torch.randn( 1 , 5 )

print ( "Indexed Matrix:\n" ,y.index_add( 1 ,index2,ten))

print ( "Printing Indexed Matrix again:\n" ,y)

Output:

Indexed Matrix: tensor([[-0.2811, -1.0776, 2.2697, 1.0000, 1.0000], [-0.2811, -1.0776, 2.2697, 1.0000, 1.0000], [-0.2811, -1.0776, 2.2697, 1.0000, 1.0000], [-0.2811, -1.0776, 2.2697, 1.0000, 1.0000], [-0.2811, -1.0776, 2.2697, 1.0000, 1.0000]]) Printing Indexed Matrix again: tensor([[1., 1., 1., 1., 1.], [1., 1., 1., 1., 1.], [1., 1., 1., 1., 1.], [1., 1., 1., 1., 1.], [1., 1., 1., 1., 1.]])

3.index_copy_: Copies the elements of a given tensor to the input tensor by selecting the indices in the order given in ‘index’.

Syntax:

index_copy_(dim,index,tensor)---> Tensor

Parameters:

Example 4: Here elements of ‘a’ is being replaced by ‘t1’ tensor in an order given in the ‘index3’.

Python3

import torch

a = torch.ones( 4 , 4 )

t1 = torch.randn( 2 , 4 )

index3 = torch.tensor([ 1 , 3 ])

a.index_copy_( 0 ,index3,t1)

Output:

tensor([[ 1.0000, 1.0000, 1.0000, 1.0000], [-0.1918, -1.2089, 0.3229, -0.1831], [ 1.0000, 1.0000, 1.0000, 1.0000], [ 0.7521, 0.8424, -0.8698, -0.3908]])

Example 5: In the below example we will receive an error. This is because the dimth dimension of the tensor is not equal to the length of the index. So we have to keep in mind that dimth dimension of the tensor must have the same size as the length of the index.

Python3

import torch

y = torch.ones( 5 , 5 )

index1 = torch.tensor([ 0 , 1 , 2 , 3 , 4 ])

te = torch.tensor([[ 1 , 3 , 5 , 7 , 9 ],[ 1 , 3 , 5 , 7 , 9 ],[ 1 , 3 , 5 , 7 , 9 ]],dtype = torch.float32)

y.index_copy_( 1 ,index1,te)

Output:

RuntimeError Traceback (most recent call last) in 1 y=torch.ones(5,5) 2 index1=torch.tensor([0,1,2,3,4]) ----> 3 y.index_copy_(1,index1,te)

RuntimeError: index_copy_(): Source/destination tensor must have same slice shapes. Destination slice shape: 5 at dimension 1 and source slice shape: 3 at dimension 0.

Example 6: In this, we will copy the given tensor to the self tensor keeping in mind that dimth dimension of the tensor must have the same size as the length of the index.

Python3

import torch

b = torch.ones( 4 , 4 )

t2 = torch.randn( 4 , 2 )

index4 = torch.tensor([ 0 , 1 ])

b.index_copy_( 1 ,index4,t2)

Output:

tensor([[-0.3964, -0.3859, 1.0000, 1.0000], [ 2.6910, -0.9394, 1.0000, 1.0000], [ 0.3591, -0.2262, 1.0000, 1.0000], [ 1.2102, -0.8340, 1.0000, 1.0000]])

4.index_copy: This is the out-of-place index-based operation for replacing elements of input tensor with a given tensor. Syntax, parameters are the same as the above.

5.index_fill_: ‘Val’ value is filled with the elements of ‘x’ along with the order of indices given in the vector ‘index’.

Syntax:

index_fill_(dim, index, val) → Tensor

Parameters:

Example 7: In this example, we will declare a tensor with random elements and then fill it with ‘4’ along with the given indices.

Python3

import torch

c = torch.randn( 4 , 4 )

index5 = torch.tensor([ 0 , 2 ])

c.index_fill_( 0 ,index5, 4 )

print (c)

Output:

tensor([[ 4.0000, 4.0000, 4.0000, 4.0000], [ 0.4762, 0.0056, 0.3258, 1.1345], [ 4.0000, 4.0000, 4.0000, 4.0000], [-0.1490, -0.6543, 0.9755, 1.8087]])

Example 8: Similarly we perform a filling operation along with the columns.

Python3

d = torch.randn( 5 , 5 )

d.index_fill( 1 ,index5, 2 )

print (d)

Output:

tensor([[ 0.5978, -1.2461, -0.8794, -1.0175, 0.8938], [-0.6374, 1.0848, 0.1291, 0.6658, 0.3081], [-0.9686, -0.8212, -0.5223, -0.3208, -1.7718], [-0.1153, -1.2552, -0.4119, -1.1293, 0.2266], [ 1.2610, 0.2618, -1.5528, 0.7805, 1.3730]])

6. index_fill: This is the out of place index-based operation for filling the elements with tensor with ‘val’. Syntax, parameters are the same as the above.

7.index_put_: This operation puts the value of ‘val’ into the self tensor using the indices of the given ‘index’.

Syntax:

index_put_(indices, values, accumulate=False) → Tensor

Parameters:

Example 9: Here we take the target vector and replaces the values of value tensor as mentioned in the

Python3

import torch

target = torch.zeros([ 4 , 4 ])

indices = torch.LongTensor([[ 0 , 1 ],[ 1 , 2 ],[ 3 , 1 ],[ 1 , 0 ]])

value = torch.ones(indices.shape[ 0 ])

target.index_put_( tuple (indices.t()), value)

Output:

tensor([[0., 1., 0., 0.], [1., 0., 1., 0.], [0., 0., 0., 0.], [0., 1., 0., 0.]])

Note: We have to take the transpose of the indices tensor or else an error will occur.

Example 10: In this example, we keep the accumulate function as true which means the elements in value are added to the target.

Python3

e = torch.ones([ 4 , 4 ])

indices2 = torch.LongTensor([[ 0 , 1 ],[ 0 , 1 ],[ 2 , 1 ]])

value2 = torch.zeros(indices2.shape[ 0 ])

e.index_put_( tuple (indices2.t()),value2,accumulate = True )

Output: tensor([[1., 1., 1., 1.], [1., 1., 1., 1.], [1., 1., 1., 1.], [1., 1., 1., 1.]])

8. index_fill: This is the out of place version of index_fill_. Syntax, parameters are the same as above.

9. index_select: A tensor is returned with indices as mentioned, by selecting from the target tensor.

Syntax:

torch.index_select(input, dim, index, out=None)

Parameters:

Example 11: We pick up 0,1 indices of the tensor ‘m’ along dim=0 i.e. rows and print the output.

Python3

import torch

m = torch.randn( 3 , 4 )

print ( 'Original matrix:\n' ,m)

indices = torch.tensor([ 0 , 1 ])

print ( "Indexed Matrix:\n" ,torch.index_select(m, 0 , indices))

Output:

Original matrix: tensor([[ 0.2008, -0.2637, 2.1216, -0.2892], [-0.4059, -1.6054, -2.5022, -0.2912], [-0.3246, 0.4751, -0.1018, -0.6707]]) Indexed Matrix: tensor([[ 0.2008, -0.2637, 2.1216, -0.2892], [-0.4059, -1.6054, -2.5022, -0.2912]])

Reference: https://pytorch.org/docs/stable/tensors.html