Part1 Preface
Hello everyone, my name is ABC_123 . I once spent time building various web servers and database environments to study and analyze logs. The tools I used ranged from using the system’s own commands to analyze logs, to automated 360 star maps, to later Logparser, ELK (ElastiSearch, Kibana, Logstash )etc. In the end, I found that the commercial version of Splunk is easier to use. I personally think this software is much easier to use than ELK. However, ELK is free and open source and can be packaged into products, so it is sought after by everyone. On the contrary, Splunk is expensive. , promotion is very limited . Splunk can be used well to cope with various existing log analysis scenarios, and it can be called an artifact.
Splunk is a commercial tool written by Americans (ELK is also made by Americans). It began to develop and update in 2004. It can collect log data, performance data, network packets and other data generated by equipment and software, and then index it. , investigation, monitoring, visualization, etc. Of course, you can also use it to assist the blue team’s log analysis work. Many users also use it for product marketing analysis. The new version of Splunk can also combine previous data to analyze customer visits in the future. predict.
Welcome to follow my public account "ABC123 Security Research Laboratory", which is 99% original and does not post hydrology or advertisements .
Part2 Splunk usage introduction
Splunk’s search syntax
Next, let’s briefly introduce Splunk’s search syntax:
Supports Boolean operators ( AND/OR/NOT ), must be uppercase
status=200 NOT action=purchase
status=200 action!=purchase
500 select finds all events containing the words "500" and "select"
500 sel* finds all events that contain words starting with "500" and "sel"
status=500 use= returns accurate results
head n //Return the first n
tail n //Return the next n
top //Display the most common/most frequent values of the field
rare //Display the least common value of the field
limit //Limit the query, such as: limit 5, Limit the first 5 results
rename xx as zz //Set the alias for the xx field to zz, use it between multiple fields to separate
fields //Keep or delete the fields in the search results. fields – xx deletes the xx field, retaining it does not require the – symbol
Fields can be inserted in the brackets of stats count() , which mainly counts events.
stats dc() distinct count. Statistics of unique values after deduplication.
stats values() lists the field contents in brackets after deduplication.
stats avg() averages value
As shown in the picture below, this picture was modified from a very old ppt. You can intuitively see the Splunk interface and how to use it.
The following is an example of Splunk's dashboard interface. In this interface, blue team analysts can put together analysis charts of various log analyses, and the display is very intuitive and beautiful .
Import log files
There are many ways to import log files into Splunk. I am more accustomed to importing logs in the following way. The specific operations are as follows: Settings-->Data Input-->Files and Directories-->New Local Files and Directories
Then click the " Browse " button next to " File or Directory " and select a folder on your local machine, such as c:\log111
Later, just put the log files that need to be analyzed into this folder, and Splunk will automatically import them.
After placing the log files in the above folder, open the main interface of Splunk. There is a row of green histograms in the middle, showing the number of web visits in each time period. What needs to be focused on here is the sudden high number in the histogram . The columnar part is most likely a time period when attackers perform frequent operations .
If you need to perform log analysis on the specified log file, you only need to enter the following command: source="C:\\log1111\\45_secure.txt"
Analyze SQL injection data-free behavior
Using the following statement, you can quickly search for SQL injection attack behaviors. In order to reduce false positives, you can use the AND statement to splice IISCode!=404 to exclude search results with a response code of 404. As shown in the figure below, combined with the histogram, we can know that SQL injection attacks are mainly concentrated within the period of about an hour shown in the histogram. It is speculated that the attacker discovered the SQL injection vulnerability and carried out the attack. Only one select keyword is used here, because if an attacker wants to use a SQL injection vulnerability to remove data, the select keyword is almost necessary .
index=_* OR index=* sourcetype="SQL_Injectionex" select AND iisCode!=404
Analyze XSS attack behavior
Using the following statement, you can quickly analyze XSS attack behavior and filter the log results with a response code of 200 through the logical operator AND. Splunk can support the analysis of tag characters such as < and > . When using it, you need to add double quotes to the keywords .
index=_* OR index=* sourcetype="SQL_Injectionex" "<script>" OR "alert" OR "confirm" AND iisCode=200
Combined with Splunk's histogram, we can know that XSS attack behavior is concentrated within the 1-hour period shown in the histogram.
Analysis of causes of slow website response
The customer discovered that a web server on the intranet accessed a special card every 13:00 p.m. Through Splunk's log analysis, it was found that there was a peak access of about an hour (1 hour in the figure), and the accessed URL Almost all are in the /shop/rexsearch.asp path. It is guessed that frequent access to the search function has caused the server to be overloaded.
Next, we counted the number of visits to /shop/rexsearchp.asp and found that there were 22,418 POST requests and only 2 GET requests.
I will tell you about this case in detail later when I have time. Finally, it was found that it was caused by a vulnerability scanner on the intranet. Every 13 o'clock in the afternoon, the web server will be automatically scanned for vulnerabilities. The scanner will scan the search page for vulnerabilities . Frequent detection causes the server to be overloaded .
Analyze SSH password blasting events
As shown in the figure below, in order to analyze the SSH password enumeration event, directly fill in the " authentication " keyword in the search box . After Splunk completes the analysis, a histogram is given. Each histogram represents the number of SSH password attacks within 1 minute. . According to Splunk's histogram and appropriately adjusting the timeline length, we can intuitively see that the brute-force cracking of SSH passwords in this log is mainly concentrated in the minute from 12:21 to 12:22 . The attacker conducted a total of 51,158 Password enumeration operation.
table table display command
Through the table command, Splunk's search results can be organized into a table format, and the titles of each column field of the table can be customized. What's more valuable is that Splunk's search statements support Chinese and Chinese titles can be set .
source="C:\\log1111\\45_secure.txt" authentication|rename rhost as attack source IP | stats count as "source IP attack count" by attack source IP | table attack source IP source IP attack count | sort - source IP Number of attacks
As shown in the figure below, you can save the search results directly, or export the malicious IP directly, which is very convenient.
Extract ip address field
Next, we need to count which source IP addresses the attacker used, so we need to extract the source IP address as a field from the log. Splunk conveniently provides us with the function of extracting fields. Let’s see the next operation:
First, click the " Extract New Field " button and select a row of logs as a sample for field analysis and extraction.
Next click " Next " and select the " Regular Expression " button.
Next, come to the " Select Field " function, use the left mouse to drag and select an IP address. Splunk will automatically generate a regular expression for us. Here, we name the field secureIP .
After the addition was completed, we found that Splunk automatically prepared regular expression statements for us, which can be modified as needed.
Next, enter the query statement "authentication" | top secureRhost | sort - secureRhost , and you can intuitively see which IP addresses have the highest number of explosions. Here you need to pay attention to the usage of the pipe character |, output on the left and input on the right.
Next, the distribution of IP addresses is displayed in a chart. Enter the command "authentication" | iplocation secureRhost | geostats count by secureRhost globallimit=0 . You can see that the attacker is likely to use a botnet or proxy pool in about a minute. During this time, more than 60,000 account enumeration operations were performed .
Use of iplocation command
Entering this command can automatically add fields such as City, Country, Region, lat, and lon, and display the country, region, longitude and latitude corresponding to the IP address in the log, which is very convenient.
source="C:\\log1111\\45_secure.txt" authentication |stats count by rhost | iplocation rhost | search Country!="China"
Dashboard graphic display function
Through Splunk, various analysis charts can be summarized on the " dashboard " for display. The following is a demonstration I made, which summarizes three analysis charts into one interface. The specific usage method will be discussed later.
Part3 Summary
1. The functions of Splunk are very complicated. It is difficult to explain Splunk clearly in one article. In the future, ABC_123 will continue to share how to use Splunk, a log analysis tool, and will also publish a video tutorial, so stay tuned.
2. Follow the official account and reply "2 022" to get the PDF e-book download address of "2022 ABC123 Official Account Annual" .
The public account focuses on sharing network security technology, including APT event analysis, red team attack and defense, blue team analysis, penetration testing, code audit, etc. One article per week, 99% original, so stay tuned.
Contact me: 0day123abc#gmail.com(replace # with @)