Recommended system notes (eleven): pits encountered when using the coo_matrix function

Usage of coo_matrix function

         Since the usage is not the focus of the introduction, here is a brief introduction:

        Format: coo_matrix((data, (row, col)), shape=(, ))
        The code example is as follows:

from scipy.sparse import coo_matrix, hstack
import pandas as pd

da = pd.read_csv('data/dummy/data.csv')

num = len(da['user']) # 列的长度
rows = list(range(num))   # list(range(5)):[0,1,2,3,4]
num1 = coo_matrix(([1] * num, (rows, da['user'])), shape=(num, 3)).toarray()
print(num1)


The content of the data.csv file is: insert image description here

operation result:

operation result

For detailed usage, please refer to: How to use coo_matrix

 

TypeError: ‘coo_matrix‘ object is not subscriptable

        The above error occurs when trying to index coo_matrix because the sparse matrix coo_matrix does not directly support slice operations. Slicing operations essentially retrieve elements based on a series of subscript indices. Therefore, the sparse matrix coo_matrix does not directly support obtaining elements according to the subscript index.

        So how should we solve it? I need to randomly remove some columns and rows of the coo_matrix during the recommendation system task, but I can’t perform matrix multiplication to set a row to zero by indexing, so I just set the corresponding data to zero directly. up.

        But the general solution is to use the function .toarray to convert it into an array to perform indexing and multiplication, namely:

          coo_matrix((data, (row, col)), shape=(M,N)).toarray()

        However, if the array is too large, using numpy's dot algorithm may cause memory overflow. Memory overflow has the following solutions:

        1. Modify numpy to 64bit version.

        2. Modify the precision of float64 to low precision such as float16, float32, etc.

        3. Go to the console to modify the available virtual memory.

Guess you like

Origin blog.csdn.net/qq_46006468/article/details/126129595