Phenomenon review :
You can use a custom data set for network training under the MindSpore dynamic graph, but when using a custom data set, the following error will occur if you are not careful:
When this error occurs, I just see that a pointer is empty, but the python call stack is in the data set. Then open the corresponding custom dataset script:
/home/cjj/models/official/cv/warpctc/src/dataset.py(57)
It is found that there is an image += 1.0 tensor operation here. So after deleting the operation, it was found that the program was running normally.
Root cause analysis :
The root cause of this error is deeply hidden. Because the MindSpore framework only supports single-threaded operations for the execution of a single operator, but the Tensor operation is used in the custom data set __getitem__ function, that is, it will be transferred to the operator of the framework for execution. , because the processing of the data set uses multi-threaded operations, the overall execution order is disordered, and a null pointer error occurs. So when you see a null pointer error and the error is in generator.cc, it means that the operation of Tensor is used incorrectly in the data set.
Solution :
1. According to the prompt call stack information, find the error location in the custom dataset script.
2. Change the Tensor operation in the custom dataset to use native numpy for calculation.