I. Background
Recently, a giant slow server response A development environment, look at the situation on the server. It found that 400% cpu continued, realizing that this server has been compromised. Here is my idea of the whole investigation, we want to help.
Second, the idea of mining investigation virus
2.1 Viewing abnormal process
Log server through the top command, found more than 300% cpu usage of a process, and this process is very strange name, is a random string, shots are as follows:
2.2 kill off the process
Use the command kill -9 27596 end of the process, discovered cpu share immediately dropped to a reasonable level, and the overall response rate was significantly upgrade the server to determine that the process pounding ghost.
2.3 and up mining virus
- But over a period of time after the emergence of the slow response of the server, and perform the above operation, in order to determine the timing of the process is the background to start, once again check the system scheduled tasks, use the command crontab -l, indeed found there is a timed task, as follows:
[root@test222 ~]# crontab -l
0 * * * * ~/.systemd-login
[root@rabbit03 ~]#
- Once again, top view process, appeared in the same random string of abnormal process, began to see this shell specifically what to do, use the more command to view:
- This line will then be decoded into base64 decoding system, to give the following codes:
- Which found that the script is to obtain execute code from a remote server, and then randomly start time, it is a typical script injection invasion.
Third, remove the virus mining
3.1 normal way of thinking
- Delete the scheduled task, mining script, but what did not, over time the virus will still mining ourselves up, leading to surge cpu
crontab -e //编辑并删除定时任务
rm -rf ~/.systemd-login
3.2 turn off some of the less frequently used services
- We suspect that by tomcat application (tomcat8 hackers use loopholes) attack coming in, so we put all the service stopped tomcat applications, still no use, cpu over time or will surge
System version 3.3 upgrade, only open ports must be open and modify the default ssh port number
1. mining virus has invaded the system will start up its own timing, so I decided to reset the system
- We made the system with a virus mining snapshot. Then reset the system. Bought a dial to select the snapshot as the initial data, and mount to the machine, the copy need to copy everything out and then restart the service, and finally the release of the data from the disk snapshot of a virus mining
2.我们加强了对安全组的限制，只开放一些必要的端口号出来，对外部攻击保持敬畏
3.修改默认ssh端口并配置更复杂的密码，防止外部的全网扫描和暴力破解
4.这个问题咨询过阿里的安全专家，他们给的答复是可能是利用Consul RCE漏洞传播的挖矿木马，但是我们的机器并没有consul相关的服务，最后他们推荐使用阿里云的安全产品WAF进行安全加固。
