文章目录
- `nvidia-smi`查看所有GPU的简要信息状态
- `nvidia-smi –i xxx`指定查看某个GPU
- `nvidia-smi –l xxx`动态刷新信息（默认5s刷新一次），按Ctrl+C停止，可指定刷新频率，以秒为单位
- `nvidia-smi –f xxx` 将查询的信息输出到具体的文件中，不在终端显示
- `nvidia-smi -q`查看所有GPU的详细信息状态
- `nvidia-smi –q –u`显示单元而不是GPU的属性
- `nvidia-smi –q –i xxx`指定具体的GPU或unit信息
- `nvidia-smi –q –f xxx`将查询的信息输出到具体的文件中，不在终端显示
- `nvidia-smi –q –x`将查询的信息以xml的形式输出
- `nvidia-smi -q –d xxx`指定显示GPU卡某些信息，xxx参数可以为MEMORY, UTILIZATION, ECC, TEMPERATURE, POWER,CLOCK, COMPUTE, PIDS, PERFORMANCE, SUPPORTED_CLOCKS, PAGE_RETIREMENT,ACCOUNTING
- `nvidia-smi –q –l xxx`动态刷新信息，按Ctrl+C停止，可指定刷新频率，以秒为单位
- `nvidia-smi --query-gpu=gpu_name,gpu_bus_id,vbios_version --format=csv`选择性查询选项，可以指定显示的属性选项
- 设备修改选项
- `nvidia-smi –pm 0/1`设置持久模式：0/DISABLED,1/ENABLED
- `nvidia-smi –e 0/1`切换ECC支持：0/DISABLED, 1/ENABLED
- `nvidia-smi –p 0/1`重置ECC错误计数：0/VOLATILE, 1/AGGREGATE
- `nvidia-smi –c`设置计算应用模式：0/DEFAULT,1/EXCLUSIVE_PROCESS,2/PROHIBITED
- `nvidia-smi –r`GPU复位
- `nvidia-smi –vm`设置GPU虚拟化模式
- `nvidia-smi –ac xxx,xxx`设置GPU运行的工作频率。e.g. nvidia-smi –ac2000,800
- `nvidia-smi –rac`将时钟频率重置为默认值
- `nvidia-smi –acp 0/1`切换-ac和-rac的权限要求，0/UNRESTRICTED, 1/RESTRICTED
- `nvidia-smi –pl`指定最大电源管理限制（瓦特）
- `nvidia-smi –am 0/1`启用或禁用计数模式，0/DISABLED,1/ENABLED
- `nvidia-smi –caa`清除缓冲区中的所有已记录PID，0/DISABLED,1/ENABLED
- `nvidia-smi dmon`设备监控命令，以滚动条形式显示GPU设备统计信息。（略）
nvidia-smi查看所有GPU的简要信息状态
cd C:\Program Files\NVIDIA Corporation\NVSMI
nvidia-smi
结果：
Microsoft Windows [版本 10.0.16299.248]
(c) 2017 Microsoft Corporation。保留所有权利。
C:\Users\SIQI>cd C:\Program Files\NVIDIA Corporation\NVSMI
C:\Program Files\NVIDIA Corporation\NVSMI>nvidia-smi
Wed Jan 15 16:32:13 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 441.08 Driver Version: 441.08 CUDA Version: 10.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GT 710 WDDM | 00000000:05:00.0 N/A | N/A |
| 50% 35C P8 N/A / N/A | 262MiB / 2048MiB | N/A Default |
+-------------------------------+----------------------+----------------------+
| 1 GeForce GTX 108... WDDM | 00000000:0E:00.0 Off | N/A |
| 0% 36C P8 13W / 280W | 149MiB / 11264MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 Not Supported |
| 1 14724 C+G ...ense SDK 2.0\tools\realsense-viewer.exe N/A |
+-----------------------------------------------------------------------------+
C:\Program Files\NVIDIA Corporation\NVSMI>
不对啊，我只安装了CUDA8.0和9.0，我这哪来的10.2？
C:\Program Files\NVIDIA Corporation\NVSMI>nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2017 NVIDIA Corporation
Built on Fri_Sep__1_21:08:32_Central_Daylight_Time_2017
Cuda compilation tools, release 9.0, V9.0.176
C:\Program Files\NVIDIA Corporation\NVSMI>
实际跑六个摄像头轮询时的GPU使用情况：
nvidia-smi –i xxx指定查看某个GPU
示例：
查看GT 710
nvidia-smi –i 0
查看1080ti
nvidia-smi –i 1
nvidia-smi –l xxx动态刷新信息（默认5s刷新一次），按Ctrl+C停止，可指定刷新频率，以秒为单位
就是不断刷新打印信息，如：
3秒打印一次信息
nvidia-smi –l 3
nvidia-smi –f xxx 将查询的信息输出到具体的文件中，不在终端显示
不过试了一下不知道怎么输出。。。显示失败
后来试着输出到D盘，成功了。。。
在D盘新建一个名为print的文件夹，将文件输出到里面，也成功了：
nvidia-smi -q查看所有GPU的详细信息状态
Microsoft Windows [版本 10.0.16299.248]
(c) 2017 Microsoft Corporation。保留所有权利。
C:\Users\SIQI>cd C:\Program Files\NVIDIA Corporation\NVSMI
C:\Program Files\NVIDIA Corporation\NVSMI>nvidia-smi -q
==============NVSMI LOG==============
Timestamp : Wed Jan 15 17:20:51 2020
Driver Version : 441.08
CUDA Version : 10.2
Attached GPUs : 2
GPU 00000000:05:00.0
Product Name : GeForce GT 710
Product Brand : GeForce
Display Mode : N/A
Display Active : N/A
Persistence Mode : N/A
Accounting Mode : N/A
Accounting Mode Buffer Size : N/A
Driver Model
Current : WDDM
Pending : WDDM
Serial Number : N/A
GPU UUID : GPU-29317d9c-e1b8-0f8d-1c0e-3ca99f2dbf3d
Minor Number : N/A
VBIOS Version : 80.28.A6.00.2F
MultiGPU Board : N/A
Board ID : N/A
GPU Part Number : N/A
Inforom Version
Image Version : N/A
OEM Object : N/A
ECC Object : N/A
Power Management Object : N/A
GPU Operation Mode
Current : N/A
Pending : N/A
GPU Virtualization Mode
Virtualization Mode : N/A
Host VGPU Mode : N/A
IBMNPU
Relaxed Ordering Mode : N/A
PCI
Bus : 0x05
Device : 0x00
Domain : 0x0000
Device Id : 0x128B10DE
Bus Id : 00000000:05:00.0
Sub System Id : 0x8C931462
GPU Link Info
PCIe Generation
Max : N/A
Current : N/A
Link Width
Max : N/A
Current : N/A
Bridge Chip
Type : N/A
Firmware : N/A
Replays Since Reset : 0
Replay Number Rollovers : 0
Tx Throughput : N/A
Rx Throughput : N/A
Fan Speed : 50 %
Performance State : P8
Clocks Throttle Reasons : N/A
FB Memory Usage
Total : 2048 MiB
Used : 277 MiB
Free : 1771 MiB
BAR1 Memory Usage
Total : N/A
Used : N/A
Free : N/A
Compute Mode : Default
Utilization
Gpu : N/A
Memory : N/A
Encoder : N/A
Decoder : N/A
Encoder Stats
Active Sessions : N/A
Average FPS : N/A
Average Latency : N/A
FBC Stats
Active Sessions : N/A
Average FPS : N/A
Average Latency : N/A
Ecc Mode
Current : N/A
Pending : N/A
ECC Errors
Volatile
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Aggregate
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Retired Pages
Single Bit ECC : N/A
Double Bit ECC : N/A
Pending Page Blacklist : N/A
Temperature
GPU Current Temp : 45 C
GPU Shutdown Temp : N/A
GPU Slowdown Temp : N/A
GPU Max Operating Temp : N/A
Memory Current Temp : N/A
Memory Max Operating Temp : N/A
Power Readings
Power Management : N/A
Power Draw : N/A
Power Limit : N/A
Default Power Limit : N/A
Enforced Power Limit : N/A
Min Power Limit : N/A
Max Power Limit : N/A
Clocks
Graphics : N/A
SM : N/A
Memory : N/A
Video : N/A
Applications Clocks
Graphics : 954 MHz
Memory : 800 MHz
Default Applications Clocks
Graphics : 954 MHz
Memory : 800 MHz
Max Clocks
Graphics : N/A
SM : N/A
Memory : N/A
Video : N/A
Max Customer Boost Clocks
Graphics : N/A
Clock Policy
Auto Boost : N/A
Auto Boost Default : N/A
Processes : N/A
GPU 00000000:0E:00.0
Product Name : GeForce GTX 1080 Ti
Product Brand : GeForce
Display Mode : Disabled
Display Active : Disabled
Persistence Mode : N/A
Accounting Mode : Disabled
Accounting Mode Buffer Size : 4000
Driver Model
Current : WDDM
Pending : WDDM
Serial Number : N/A
GPU UUID : GPU-f8f659c0-b441-5918-e7a5-5afa44563670
Minor Number : N/A
VBIOS Version : 86.02.40.00.1A
MultiGPU Board : No
Board ID : 0xe00
GPU Part Number : N/A
Inforom Version
Image Version : G001.0000.01.04
OEM Object : 1.1
ECC Object : N/A
Power Management Object : N/A
GPU Operation Mode
Current : N/A
Pending : N/A
GPU Virtualization Mode
Virtualization Mode : None
Host VGPU Mode : N/A
IBMNPU
Relaxed Ordering Mode : N/A
PCI
Bus : 0x0E
Device : 0x00
Domain : 0x0000
Device Id : 0x1B0610DE
Bus Id : 00000000:0E:00.0
Sub System Id : 0x36021462
GPU Link Info
PCIe Generation
Max : 3
Current : 3
Link Width
Max : 16x
Current : 16x
Bridge Chip
Type : N/A
Firmware : N/A
Replays Since Reset : 0
Replay Number Rollovers : 0
Tx Throughput : 160000 KB/s
Rx Throughput : 2797000 KB/s
Fan Speed : 38 %
Performance State : P2
Clocks Throttle Reasons
Idle : Not Active
Applications Clocks Setting : Not Active
SW Power Cap : Not Active
HW Slowdown : Not Active
HW Thermal Slowdown : Not Active
HW Power Brake Slowdown : Not Active
Sync Boost : Not Active
SW Thermal Slowdown : Not Active
Display Clock Setting : Not Active
FB Memory Usage
Total : 11264 MiB
Used : 9268 MiB
Free : 1996 MiB
BAR1 Memory Usage
Total : 256 MiB
Used : 229 MiB
Free : 27 MiB
Compute Mode : Default
Utilization
Gpu : 43 %
Memory : 20 %
Encoder : 0 %
Decoder : 0 %
Encoder Stats
Active Sessions : 0
Average FPS : 0
Average Latency : 0
FBC Stats
Active Sessions : 0
Average FPS : 0
Average Latency : 0
Ecc Mode
Current : N/A
Pending : N/A
ECC Errors
Volatile
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Aggregate
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Retired Pages
Single Bit ECC : N/A
Double Bit ECC : N/A
Pending Page Blacklist : N/A
Temperature
GPU Current Temp : 64 C
GPU Shutdown Temp : 96 C
GPU Slowdown Temp : 93 C
GPU Max Operating Temp : N/A
Memory Current Temp : N/A
Memory Max Operating Temp : N/A
Power Readings
Power Management : Supported
Power Draw : 250.52 W
Power Limit : 280.00 W
Default Power Limit : 280.00 W
Enforced Power Limit : 280.00 W
Min Power Limit : 140.00 W
Max Power Limit : 330.00 W
Clocks
Graphics : 1949 MHz
SM : 1949 MHz
Memory : 5005 MHz
Video : 1645 MHz
Applications Clocks
Graphics : N/A
Memory : N/A
Default Applications Clocks
Graphics : N/A
Memory : N/A
Max Clocks
Graphics : 1974 MHz
SM : 1974 MHz
Memory : 5505 MHz
Video : 1620 MHz
Max Customer Boost Clocks
Graphics : N/A
Clock Policy
Auto Boost : N/A
Auto Boost Default : N/A
Processes
Process ID : 1448
Type : C
Name : D:\20191031_tensorflow_yolov3\python\python.exe
Used GPU Memory : Not available in WDDM driver model
C:\Program Files\NVIDIA Corporation\NVSMI>
我们发现，最下面打印进程占用GPU容量时，显示Not available in WDDM driver model，可参考如下解释：
Not available in WDDM driver model => WDDM stand for Windows Display Driver Model. You can switch to TCC and obtain the information with the command: nvidia-smi -dm 1, however this operation can only performed if you do not have any display attached to the GPU. So... It's mostly impossible...
在WDDM驱动程序模型中不可用=> WDDM代表Windows Display Driver Model。 您可以使用以下命令切换到TCC并获取信息：nvidia-smi -dm 1，但是仅当您未在GPU上连接任何显示器时，才能执行此操作。 所以...这几乎是不可能的...
By the way, don't worry about a high memory usage, Tensorflow reserve as much GPU memory as it can to speed up its processes. If you prefer a finer grained control on the memory taken use the following (it may slow down a little bit your computations):
顺便说一下，不用担心内存使用率高，Tensorflow会保留尽可能多的GPU内存，以加快处理速度。 如果您希望对所使用的内存进行更细粒度的控制，请使用以下命令（它可能会降低您的计算速度）：
# 动态申请显存
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
sess = tf.Session(config=config)
You can create a dual boot on Ubuntu or just forget about this.
您可以在Ubuntu上创建双重引导，也可以忽略它。
nvidia-smi –q –u显示单元而不是GPU的属性
示例：
C:\Program Files\NVIDIA Corporation\NVSMI>nvidia-smi -q -u
==============NVSMI LOG==============
Timestamp : Thu Jan 16 09:16:33 2020
Driver Version : 441.08
CUDA Version : 10.2
HIC Info : N/A
Attached Units : 0
C:\Program Files\NVIDIA Corporation\NVSMI>
nvidia-smi –q –i xxx指定具体的GPU或unit信息
示例：
查看1080ti详细信息
C:\Program Files\NVIDIA Corporation\NVSMI>nvidia-smi -q -i 1
==============NVSMI LOG==============
Timestamp : Thu Jan 16 09:25:13 2020
Driver Version : 441.08
CUDA Version : 10.2
Attached GPUs : 2
GPU 00000000:0E:00.0
Product Name : GeForce GTX 1080 Ti
Product Brand : GeForce
Display Mode : Disabled
Display Active : Disabled
Persistence Mode : N/A
Accounting Mode : Disabled
Accounting Mode Buffer Size : 4000
Driver Model
Current : WDDM
Pending : WDDM
Serial Number : N/A
GPU UUID : GPU-f8f659c0-b441-5918-e7a5-5afa44563670
Minor Number : N/A
VBIOS Version : 86.02.40.00.1A
MultiGPU Board : No
Board ID : 0xe00
GPU Part Number : N/A
Inforom Version
Image Version : G001.0000.01.04
OEM Object : 1.1
ECC Object : N/A
Power Management Object : N/A
GPU Operation Mode
Current : N/A
Pending : N/A
GPU Virtualization Mode
Virtualization Mode : None
Host VGPU Mode : N/A
IBMNPU
Relaxed Ordering Mode : N/A
PCI
Bus : 0x0E
Device : 0x00
Domain : 0x0000
Device Id : 0x1B0610DE
Bus Id : 00000000:0E:00.0
Sub System Id : 0x36021462
GPU Link Info
PCIe Generation
Max : 3
Current : 3
Link Width
Max : 16x
Current : 16x
Bridge Chip
Type : N/A
Firmware : N/A
Replays Since Reset : 0
Replay Number Rollovers : 0
Tx Throughput : 302000 KB/s
Rx Throughput : 0 KB/s
Fan Speed : 35 %
Performance State : P2
Clocks Throttle Reasons
Idle : Not Active
Applications Clocks Setting : Not Active
SW Power Cap : Not Active
HW Slowdown : Not Active
HW Thermal Slowdown : Not Active
HW Power Brake Slowdown : Not Active
Sync Boost : Not Active
SW Thermal Slowdown : Not Active
Display Clock Setting : Not Active
FB Memory Usage
Total : 11264 MiB
Used : 9268 MiB
Free : 1996 MiB
BAR1 Memory Usage
Total : 256 MiB
Used : 229 MiB
Free : 27 MiB
Compute Mode : Default
Utilization
Gpu : 42 %
Memory : 20 %
Encoder : 0 %
Decoder : 0 %
Encoder Stats
Active Sessions : 0
Average FPS : 0
Average Latency : 0
FBC Stats
Active Sessions : 0
Average FPS : 0
Average Latency : 0
Ecc Mode
Current : N/A
Pending : N/A
ECC Errors
Volatile
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Aggregate
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Texture Memory : N/A
Texture Shared : N/A
CBU : N/A
Total : N/A
Retired Pages
Single Bit ECC : N/A
Double Bit ECC : N/A
Pending Page Blacklist : N/A
Temperature
GPU Current Temp : 64 C
GPU Shutdown Temp : 96 C
GPU Slowdown Temp : 93 C
GPU Max Operating Temp : N/A
Memory Current Temp : N/A
Memory Max Operating Temp : N/A
Power Readings
Power Management : Supported
Power Draw : 267.13 W
Power Limit : 280.00 W
Default Power Limit : 280.00 W
Enforced Power Limit : 280.00 W
Min Power Limit : 140.00 W
Max Power Limit : 330.00 W
Clocks
Graphics : 1949 MHz
SM : 1949 MHz
Memory : 5005 MHz
Video : 1645 MHz
Applications Clocks
Graphics : N/A
Memory : N/A
Default Applications Clocks
Graphics : N/A
Memory : N/A
Max Clocks
Graphics : 1974 MHz
SM : 1974 MHz
Memory : 5505 MHz
Video : 1620 MHz
Max Customer Boost Clocks
Graphics : N/A
Clock Policy
Auto Boost : N/A
Auto Boost Default : N/A
Processes
Process ID : 1448
Type : C
Name : D:\20191031_tensorflow_yolov3\python\python.exe
Used GPU Memory : Not available in WDDM driver model
C:\Program Files\NVIDIA Corporation\NVSMI>
nvidia-smi –q –f xxx将查询的信息输出到具体的文件中，不在终端显示
参考上面，示例略
nvidia-smi –q –x将查询的信息以xml的形式输出
略，就是打印的格式变成xml了
nvidia-smi -q –d xxx指定显示GPU卡某些信息，xxx参数可以为MEMORY, UTILIZATION, ECC, TEMPERATURE, POWER,CLOCK, COMPUTE, PIDS, PERFORMANCE, SUPPORTED_CLOCKS, PAGE_RETIREMENT,ACCOUNTING
示例：
C:\Users\SIQI>cd C:\Program Files\NVIDIA Corporation\NVSMI
C:\Program Files\NVIDIA Corporation\NVSMI>nvidia-smi -q -d MEMORY
==============NVSMI LOG==============
Timestamp : Thu Jan 16 09:33:08 2020
Driver Version : 441.08
CUDA Version : 10.2
Attached GPUs : 2
GPU 00000000:05:00.0
FB Memory Usage
Total : 2048 MiB
Used : 370 MiB
Free : 1678 MiB
BAR1 Memory Usage
Total : N/A
Used : N/A
Free : N/A
GPU 00000000:0E:00.0
FB Memory Usage
Total : 11264 MiB
Used : 9268 MiB
Free : 1996 MiB
BAR1 Memory Usage
Total : 256 MiB
Used : 229 MiB
Free : 27 MiB
C:\Program Files\NVIDIA Corporation\NVSMI>
nvidia-smi –q –l xxx动态刷新信息，按Ctrl+C停止，可指定刷新频率，以秒为单位
xxx单位为秒
示例略
nvidia-smi --query-gpu=gpu_name,gpu_bus_id,vbios_version --format=csv选择性查询选项，可以指定显示的属性选项
可查看的属性有：timestamp，driver_version，pci.bus，pcie.link.width.current等。（可查看nvidia-smi–help-query–gpu来查看有哪些属性）
示例：
Microsoft Windows [版本 10.0.16299.248]
(c) 2017 Microsoft Corporation。保留所有权利。
C:\Users\SIQI>cd C:\Program Files\NVIDIA Corporation\NVSMI
C:\Program Files\NVIDIA Corporation\NVSMI>nvidia-smi --query-gpu=gpu_name,gpu_bus_id,vbios_version --format=csv
name, pci.bus_id, vbios_version
GeForce GT 710, 00000000:05:00.0, 80.28.A6.00.2F
GeForce GTX 1080 Ti, 00000000:0E:00.0, 86.02.40.00.1A
C:\Program Files\NVIDIA Corporation\NVSMI>
设备修改选项
可以手动设置GPU卡设备的状态选项
nvidia-smi –pm 0/1设置持久模式：0/DISABLED,1/ENABLED
nvidia-smi –e 0/1切换ECC支持：0/DISABLED, 1/ENABLED
nvidia-smi –p 0/1重置ECC错误计数：0/VOLATILE, 1/AGGREGATE
nvidia-smi –c设置计算应用模式：0/DEFAULT,1/EXCLUSIVE_PROCESS,2/PROHIBITED
nvidia-smi –rGPU复位
nvidia-smi –vm设置GPU虚拟化模式
nvidia-smi –ac xxx,xxx设置GPU运行的工作频率。e.g. nvidia-smi –ac2000,800
nvidia-smi –rac将时钟频率重置为默认值
nvidia-smi –acp 0/1切换-ac和-rac的权限要求，0/UNRESTRICTED, 1/RESTRICTED
nvidia-smi –pl指定最大电源管理限制（瓦特）
nvidia-smi –am 0/1启用或禁用计数模式，0/DISABLED,1/ENABLED
nvidia-smi –caa清除缓冲区中的所有已记录PID，0/DISABLED,1/ENABLED
nvidia-smi dmon设备监控命令，以滚动条形式显示GPU设备统计信息。（略）
参考文章1：Windows用命令查看nvidia的GPU使用情况
参考文章2：windows如何查看nvidia显卡（GPU）的利用率和温度
