problem introduction
I just installed a brand new RTX 4080GPU to speed up the training process when running machine learning scripts.
I don't see GPU usage above 10%, which probably means the code is not running. For those unfamiliar with the Task Manager view, the reason is not so obvious. The assumption I'm making is that Windows Task Manager will only show overall GPU usage.
After some thinking, I realized that the GPU memory usage is relatively high, so the GPU must actually be doing a lot of work, and then I realized that the indicator I was after was "CUDA" performance, which is not displayed at all by default, nor is it Metrics included in the main process table in Task Manager.
What the default view of Task Manager looks like
The default view in the "Performance" tab doesn't show much operations, however, I'm maxing out the GPU, especially with CUDA. It is worth noting that it can be seen that the GPU memory is generally high at 8.7GB / 31.9GB.
Task Manager view GPU usage
Looking back at the main process view, the column showing "GPU" only shows 6.5%. I at least want this column to contain metrics for all the work happening inside the graphics card. Even that doesn't show CUDA working.
Task manager process GPU usage
Add CUDA monitoring
So if we switch back to the Performance tab and select one of the graphs and right click on the dropdown, we'll see a list of other performance metrics. Let's replace the "Copy" graph with "CUDA"
Task manager specific CUDA usage
Well, now I have a confirmation of what exactly the GPU operation consumes. Currently, it can be seen that the GPU - Cuda is working at about 30%~50%
RTX4080 GPU occupancy (actual working speed)
Finally, I recommend a software to check the actual usage of GPU video memory - GPU-Z, with a download link attached
GPU-Z Downloads | TechPowerUpwww.techpowerup.com/download/gpu-z/
Its display of the GPU in action is very visible and remarkable.
GPU-Z software to view the overall situation of the hardware
GPU-Z software displays graphics memory usage