pytorch extract non-NAN values

There are functions for judging NAN values ​​in numpy and torch, but there is no function for judging non-NAN values. There is a function for judging non-null values ​​in pandas, namely df.notnull(), but the data has to be converted into the form of Dataframe, because I am not used to the Dataframe data form, so consider extracting non-NAN values ​​in the form of numpy and tensor by calling the functions that come with python instead of time-consuming for loops, and record them to avoid future reference. The specific operations are as follows.
Basic idea:
For tensor, first obtain the null value by calling the torch.isnan() function, then obtain the position of the null value and the position of all values ​​by using the torch.where() function, and finally call the interpolation function to find the difference between the two. The complement is the non-NAN position. Finally, the torch.index_select() function is called to use the index value to extract the non-NAN value. The example is as follows:

# data数据格式为tensor
a = torch.where(data)[0].numpy()
b = torch.where(torch.isnan(data))[0].numpy()
d = sorted(list(set(a).difference(set(b))
NNULL = torch.index_select(data, 0, torch.tensor(d))

For numpy arrays, it is simpler, use the torch.isnan() function to find the index of the NAN value, and finally use np.delete() to delete the NAN value of the specified index, and the rest are non-NAN values. Examples are as follows:

# data数据格式为nadarry
data_temp = torch.tensor(data)
b = torch.where(torch.isnan(data_temp))[0].numpy()
NNULL = np.delete(data, b, axis=0)

Finally, some links for reference are attached:
pytorch judges whether tensor has dirty data NaN
torch.index_select() takes elements of specified index from tensor by dimension
Pandas+Numpy processing operations of null values ​​in data: judgment, search, fill and delete
Python notes : Manipulate ndarray elements: access, delete, insert
Python methods to find the intersection, union, difference (complement) set, and symmetric difference set of two lists
python - How to get the index of the next non-NaN number in the Pandas series?
Python pandas, NaN judgment (isnull(), notnull()), NaN processing, missing processing, dropna(), fillna()

Guess you like

Origin http://43.154.161.224:23101/article/api/json?id=324340066&siteId=291194637