Overview
In the daily operation and maintenance of enterprises, there are many types of equipment, different log formats, huge log volume, and a large number of alarms. We are faced with how to store these logs in a unified manner? How to view and analyze massive logs? Traditional log devices cannot meet the needs of data access in different log formats and can further analyze the collected logs. We finally selected Honghu, which can perfectly solve these pain points.
This article collects and analyzes the logs of network devices to help O&M personnel quickly locate faults and security issues.
Monitoring target
Collect logs of network devices such as switches and firewalls in the infrastructure
Monitor login security of network devices
Monitor network device configuration security
Monitor network device interface status
Architecture description
Component description
Network equipment: switches, routers, firewalls and other equipment that support syslog mode
Honghu Platform: Vector is used for data collectors. Data sets are used to store different data. Query, used to query data in a dataset.
Data import process
The network device sends data to the Honghu platform through syslog; after Vector collects the data, it stores it in the data set to provide data for query.
Syslog data import configuration
Note that the Honghu platform needs to be able to access the Internet.
Configure Syslog data collection in Honghu
Install Vector
Check the vector version after installation to confirm that the installation is successful.
After vector is installed, when executing vector directly, the system will first look for the command in /usr/bin. If it is not in this directory, it will not be found. At this time, we need to create a link file for these commands that cannot be found and link to /usr/bin.
(For specific codes, please join the Honghu technical exchange group and obtain them from the knowledge base)
Log in to the Honghu Platform, Data Management > Create a new data set
Edit the data source name, select the data set range as the "switch" created above, and it will be enabled.
Create syslog.toml script, fields need to be adjusted
address = "0.0.0.0:514": 0.0.0.0 means receiving syslog sent by all hosts, 514 means the receiving port (syslog defaults to 514)
mode = "udp": Indicates the protocol for receiving syslog. Syslog defaults to udp)
._target_table = "switch" : Indicates the name of the dataset you created above
address = "172.20.6.111:9092": Honghu's IP and corresponding port
Run the modified syslog script, keeping it running.
(For specific codes, please join the Honghu technical exchange group and obtain them from the knowledge base)
Log in to the switch to trigger syslog (note: logging in to the switch and inputting commands will automatically trigger syslog). Log in to the Honghu platform to check whether the data is imported into the switch dataset. As shown in the figure below, the event count has shown that the data import is successful
Query the data imported into the switch dataset through vector
Network device configuration syslog
H3C switch syslog configuration
Logs need to be turned on and sent to the Honghu platform. By default, syslog sends logs through udp514 port
Use the host with the IP address 172.20.6.111 as the log host and use the loghost channel to send information (optional, the system default is
loghost channel), use local5 as the log host recording tool.
Note: You can specify the source address for sending logs here. You don’t need to specify it. Generally, for the convenience of identification, it is recommended to specify it.
Note: If you want to send logs through the management port, since the management port is configured with VPN-Instance, you need to specify VPN-Instance.
Configure output rules: allow all modules and log information with levels higher than or equal to informational to be output to the log host.
The actual configuration command I have here
Cisco switch syslog configuration
The configuration commands on Cisco network equipment are generally:
In global mode <config>
The actual configuration command I have here
Field extraction
Since H3C switches have their own specific log format, Honghu will not extract all fields, so data must be extracted based on the log format of H3C switches. The logic of field extraction is to first create a view through SQL statements. After the view is generated, we can directly call the fields in the view, and the actual logs are still stored in the original data set.
Let’s first analyze the log format of H3C switches. The official explanation log format is as follows:
Our actual log format analysis, the fields that need to be extracted
switch_syslog, here is the name of the view that needs to be created
In the switch._time script, starting from the sixth line, switch refers to the original data set that needs to extract data. Here you need to replace it according to your data set name.
Run in high-level query, successful
In the view, you can see the view of switch_syslog we created
Let's run the search statement test, and we can see the fields we extracted in the red part
Chart display
Dashboard>New Dashboard
Creation completed
Equipment change statistics
New chart>Equipment change count statistics
Select chart type: Pie chart
Query statement: This statement can be verified in the query first to confirm that the search result is what you want.
Time range: Choose 30 days, you can adjust it according to your own situation
After generating the chart, you can view which devices have had their configuration changed and how many times in the past 30 days.
Purpose: For example, if there has been no maintenance in the past month and the device has been changed too many times, there will be abnormalities. By querying to see who has logged in and what they have done, you can determine whether these behaviors are normal and compliant.
Device login count statistics
New icon>User login count statistics
Select chart type: Bar chart
Query statement: This statement can be verified in the query first to confirm that the search result is what you want.
Time range: Choose 30 days, you can adjust it according to your own situation
After generating the chart, you can see which devices have logged in how many times in the past 30 days.
Purpose: For example, if there has been no maintenance in the past month and the device has been logged in too many times, there will be abnormalities. By querying to see who has logged in and what they have done, you can determine whether these behaviors are normal and compliant.
User login event chart
New Icon>User Login Event Chart
Select the chart type: table (since I need to display details here, I choose the table method)
Query statement: This statement can be verified in the query first to confirm that the search result is what you want.
Time range: Choose 30 days, you can adjust it according to your own situation
After generating the chart, you can view the specific time, which device, and which user logged in from which IP.
Purpose: For example, during non-maintenance or non-working hours, there are exceptions when logging into the device. By querying to see who logged in to the device during this time period, we can determine whether these behaviors are normal and compliant.
Device configuration change chart
New Icon>Device Configuration Change Chart
Select the chart type: table (since I need to display details here, I choose the table method)
Query statement: This statement can be verified in the query first to confirm that the search result is what you want.
Time range: Choose 30 days, you can adjust it according to your own situation
After generating the chart, you can view the specific time, which user, which device, and which command was executed.
Purpose: For example, changing the device configuration during non-maintenance time or non-working time, and querying to see who logged in to the device during this time period and what configurations were made, so as to determine whether these behaviors are normal and compliant.
search statement
All search statements in this article are listed below for reference
Tip: The command part of the search statement will call the extracted character segment. If the character segment is not extracted, an error will appear.
renderings
After the chart is created, we select "Grid Layout" to optimize and adjust the chart layout, and the final generated effect is as follows
