I am trying to downsize a tensor from let's say (3,3) to (1, 1), but I want to keep the original tensor:
import torch
a = torch.rand(3, 3)
a_copy = a.clone()
a_copy.resize_(1, 1)
I need requires_grad=True in my initial tensor but PyTorch forbids this me from trying to resize the copy:
a = torch.rand(3, 3, requires_grad=True)
a_copy = a.clone()
a_copy.resize_(1, 1)
Throws an error:
Traceback (most recent call last):
File "pytorch_test.py", line 7, in <module>
a_copy.resize_(1, 1)
RuntimeError: cannot resize variables that require grad
Clone and Detach
I tried to .clone() and .detach() as well:
a = torch.rand(3, 3, requires_grad=True)
a_copy = a.clone().detach()
with torch.no_grad():
a_copy.resize_(1, 1)
which gives this error instead:
Traceback (most recent call last):
File "pytorch_test.py", line 14, in <module>
a_copy.resize_(1, 1)
RuntimeError: set_sizes_contiguous is not allowed on a Tensor created from .data or .detach().
If your intent is to change the metadata of a Tensor (such as sizes / strides / storage / storage_offset)
without autograd tracking the change, remove the .data / .detach() call and wrap the change in a `with torch.no_grad():` block.
For example, change:
x.data.set_(y)
to:
with torch.no_grad():
x.set_(y)
This behaviour had been stated in the docs and #15070.
With no_grad()
So, following what they said in the error message, I removed .detach() and used no_grad() instead:
a = torch.rand(3, 3, requires_grad=True)
a_copy = a.clone()
with torch.no_grad():
a_copy.resize_(1, 1)
But it still gives me an error about grad:
Traceback (most recent call last):
File "pytorch_test.py", line 21, in <module>
a_copy.resize_(1, 1)
RuntimeError: cannot resize variables that require grad
Similar questions
I have looked at Resize PyTorch Tensor but it the tensor in that example retains all original values. I have also looked at Pytorch preferred way to copy a tensor which is the method I am using to copy the tensor.
I am using PyTorch version 1.4.0
I think you should first detach and then clone:
a = torch.rand(3, 3, requires_grad=True)
a_copy = a.detach().clone()
a_copy.resize_(1, 1)
Note: a.detach() returns a new tensor detached from the current graph (it doesn't detach a itself from the graph as a.detach_() does). But because it shares storage with a, you also should clone it. In this way, whatever you do with the a_copy won't affect a. However I am not sure why a.detach().clone() works but a.clone().detach() gives error.
Edit:
The following code also works (which is probably a better solution):
a = torch.rand(3, 3, requires_grad=True)
with torch.no_grad():
a_copy = a.clone()
a_copy.resize_(1, 1)