Computer process tool, a brief introduction to the psutil module

 

Reference from Liao Da: https://www.liaoxuefeng.com/wiki/1016959663602400/1183565811281984

 

Writing scripts in Python to simplify daily operation and maintenance is an important use of Python. In Linux, there are many system commands allows us time to monitor the status of system operation, such as ps, top, freeand so on. To obtain these system information, Python can subprocesscall through the module and get the result. But this is very troublesome, especially if you have to write a lot of parsing code.

Another good way to get system information in Python is to use psutilthis third-party module. As the name implies, psutil = process and system utilities, it can not only realize system monitoring through one or two lines of code, but also can be used across platforms. It supports Linux / UNIX / OSX / Windows, etc. It is indispensable for system administrators and operation and maintenance partners Required modules.

Install psutil

If Anaconda is installed, psutil is already available. Otherwise, you need to install via pip at the command line:

$ pip install psutil

If you encounter a Permission denied installation failure, please add sudo and try again.

Get CPU information

Let's get the CPU information first:

>>> import psutil
>>> psutil.cpu_count() # CPU逻辑数量
4 >>> psutil.cpu_count(logical=False) # CPU物理核心 2 # 2说明是双核超线程, 4则是4核非超线程 

Statistics CPU user / system / idle time:

>>> psutil.cpu_times()
scputimes(user=10963.31, nice=0.0, system=5138.67, idle=356102.45) 

Re-implement topthe CPU usage of similar commands, refreshing once per second, a total of 10 times:

>>> for x in range(10):
... print(psutil.cpu_percent(interval=1, percpu=True)) ... [14.0, 4.0, 4.0, 4.0] [12.0, 3.0, 4.0, 3.0] [8.0, 4.0, 3.0, 4.0] [12.0, 3.0, 3.0, 3.0] [18.8, 5.1, 5.9, 5.0] [10.9, 5.0, 4.0, 3.0] [12.0, 5.0, 4.0, 5.0] [15.0, 5.0, 4.0, 4.0] [19.0, 5.0, 5.0, 4.0] [9.0, 3.0, 2.0, 3.0] 

Get memory information

Use psutil to obtain physical memory and swap memory information, use:

>>> psutil.virtual_memory()
svmem(total=8589934592, available=2866520064, percent=66.6, used=7201386496, free=216178688, active=3342192640, inactive=2650341376, wired=1208852480) >>> psutil.swap_memory() sswap(total=1073741824, used=150732800, free=923009024, percent=14.0, sin=10705981440, sout=40353792) 

The returned value is an integer in bytes. As you can see, the total memory size is 8589345592 = 8 GB, 7201386496 = 6.7 GB has been used, and 66.6% is used.

The size of the swap area is 1073741824 = 1 GB.

Get disk information

You can use psutil to obtain disk partition, disk usage, and disk IO information:

>>> psutil.disk_partitions() # 磁盘分区信息
[sdiskpart(device='/dev/disk1', mountpoint='/', fstype='hfs', opts='rw,local,rootfs,dovolfs,journaled,multilabel')] >>> psutil.disk_usage('/') # 磁盘使用情况 sdiskusage(total=998982549504, used=390880133120, free=607840272384, percent=39.1) >>> psutil.disk_io_counters() # 磁盘IO sdiskio(read_count=988513, write_count=274457, read_bytes=14856830464, write_bytes=17509420032, read_time=2228966, write_time=1618405) 

It can be seen that '/'the total capacity of the disk is 998982549504 = 930 GB, which is 39.1% used. The file format is HFS, optswhich rwmeans that it can read and write, and journaledsupports log.

Get network information

psutil can get network interface and network connection information:

>>> psutil.net_io_counters() # 获取网络读写字节／包的个数
snetio(bytes_sent=3885744870, bytes_recv=10357676702, packets_sent=10613069, packets_recv=10423357, errin=0, errout=0, dropin=0, dropout=0) >>> psutil.net_if_addrs() # 获取网络接口信息 { 'lo0': [snic(family=<AddressFamily.AF_INET: 2>, address='127.0.0.1', netmask='255.0.0.0'), ...], 'en1': [snic(family=<AddressFamily.AF_INET: 2>, address='10.0.1.80', netmask='255.255.255.0'), ...], 'en0': [...], 'en2': [...], 'bridge0': [...] } >>> psutil.net_if_stats() # 获取网络接口状态 { 'lo0': snicstats(isup=True, duplex=<NicDuplex.NIC_DUPLEX_UNKNOWN: 0>, speed=0, mtu=16384), 'en0': snicstats(isup=True, duplex=<NicDuplex.NIC_DUPLEX_UNKNOWN: 0>, speed=0, mtu=1500), 'en1': snicstats(...), 'en2': snicstats(...), 'bridge0': snicstats(...) } 

To get the current network connection information, use net_connections():

>>> psutil.net_connections()
Traceback (most recent call last):
  ...
PermissionError: [Errno 1] Operation not permitted During handling of the above exception, another exception occurred: Traceback (most recent call last): ... psutil.AccessDenied: psutil.AccessDenied (pid=3847) 

You may get an AccessDeniederror because psutil also needs to go through the system interface to obtain information, and root permission is required to obtain network connection information. In this case, you can exit the Python interactive environment and sudorestart with:

$ sudo python3
Password: ******
Python 3.8 ... on darwin
Type "help", ... for more information.
>>> import psutil
>>> psutil.net_connections()
[
    sconn(fd=83, family=<AddressFamily.AF_INET6: 30>, type=1, laddr=addr(ip='::127.0.0.1', port=62911), raddr=addr(ip='::127.0.0.1', port=3306), status='ESTABLISHED', pid=3725), sconn(fd=84, family=<AddressFamily.AF_INET6: 30>, type=1, laddr=addr(ip='::127.0.0.1', port=62905), raddr=addr(ip='::127.0.0.1', port=3306), status='ESTABLISHED', pid=3725), sconn(fd=93, family=<AddressFamily.AF_INET6: 30>, type=1, laddr=addr(ip='::', port=8080), raddr=(), status='LISTEN', pid=3725), sconn(fd=103, family=<AddressFamily.AF_INET6: 30>, type=1, laddr=addr(ip='::127.0.0.1', port=62918), raddr=addr(ip='::127.0.0.1', port=3306), status='ESTABLISHED', pid=3725), sconn(fd=105, family=<AddressFamily.AF_INET6: 30>, type=1, ..., pid=3725), sconn(fd=106, family=<AddressFamily.AF_INET6: 30>, type=1, ..., pid=3725), sconn(fd=107, family=<AddressFamily.AF_INET6: 30>, type=1, ..., pid=3725), ... sconn(fd=27, family=<AddressFamily.AF_INET: 2>, type=2, ..., pid=1) ] 

Get process information

You can get detailed information of all processes through psutil:

>>> psutil.pids() # 所有进程ID
[3865, 3864, 3863, 3856, 3855, 3853, 3776, ..., 45, 44, 1, 0] >>> p = psutil.Process(3776) # 获取指定进程ID=3776，其实就是当前Python交互环境 >>> p.name() # 进程名称 'python3.6' >>> p.exe() # 进程exe路径 '/Users/michael/anaconda3/bin/python3.6' >>> p.cwd() # 进程工作目录 '/Users/michael' >>> p.cmdline() # 进程启动的命令行 ['python3'] >>> p.ppid() # 父进程ID 3765 >>> p.parent() # 父进程 <psutil.Process(pid=3765, name='bash') at 4503144040> >>> p.children() # 子进程列表 [] >>> p.status() # 进程状态 'running' >>> p.username() # 进程用户名 'michael' >>> p.create_time() # 进程创建时间 1511052731.120333 >>> p.terminal() # 进程终端 '/dev/ttys002' >>> p.cpu_times() # 进程使用的CPU时间 pcputimes(user=0.081150144, system=0.053269812, children_user=0.0, children_system=0.0) >>> p.memory_info() # 进程使用的内存 pmem(rss=8310784, vms=2481725440, pfaults=3207, pageins=18) >>> p.open_files() # 进程打开的文件 [] >>> p.connections() # 进程相关网络连接 [] >>> p.num_threads() # 进程的线程数量 1 >>> p.threads() # 所有线程信息 [pthread(id=1, user_time=0.090318, system_time=0.062736)] >>> p.environ() # 进程环境变量 {'SHELL': '/bin/bash', 'PATH': '/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:...', 'PWD': '/Users/michael', 'LANG': 'zh_CN.UTF-8', ...} >>> p.terminate() # 结束进程 Terminated: 15 <-- 自己把自己结束了 

Similar to obtaining a network connection, the process of obtaining a root user requires root permissions, .pyand sudopermissions are required when starting the Python interactive environment or files .

psutil also provides a test()function that can simulate psthe effect of the command:

$ sudo python3
Password: ******
Python 3.6.3 ... on darwin Type "help", ... for more information. >>> import psutil >>> psutil.test() USER PID %MEM VSZ RSS TTY START TIME COMMAND root 0 24.0 74270628 2016380 ? Nov18 40:51 kernel_task root 1 0.1 2494140 9484 ? Nov18 01:39 launchd root 44 0.4 2519872 36404 ? Nov18 02:02 UserEventAgent root 45 ? 2474032 1516 ? Nov18 00:14 syslogd root 47 0.1 2504768 8912 ? Nov18 00:03 kextd root 48 0.1 2505544 4720 ? Nov18 00:19 fseventsd _appleeven 52 0.1 2499748 5024 ? Nov18 00:00 appleeventsd root 53 0.1 2500592 6132 ? Nov18 00:02 configd


在使用selenium中，经常会有残余的chrome进程，虽然占用的空间不大，但一致上升的数量还是会占用内存。

# Delete more than chromedriver process 
    def del_chromedriver (self): 
        for i in psutil.process_iter (): 
            try: 
                if i.name () == 'chromedriver': 
                    i.terminate () 
            except: 
                ...

 Through process_iter, you can find out the process object more quickly, why should you add try, because my Chinese process here will have errors when reading.

Byte encoding problem, no time to study, just ignore the error directly