Logtail new features: acquisition path blacklist

background

In some scenarios, we need to collect the logs will spread across multiple directories, in order to collect configuration in a simultaneous acquisition of them, we often need to collect configuration directory is specified as the common parent directory of these directories, which is used in practice use may cause some problems. In the following directory structure, for example, we hope in a collection configuration specified collection dira/b/cdirectory *.log, you must specify the parent directory catalog will be collected admin. But such a designation, due to the acquisition logtail recursive rule subdirectories, one hand dirsunder the directory private.logdue to the compliance with the rules, will be gathering up, it would be contrary to data security; on the other hand, if the directory dire, dirfthese directories will not need to be monitored , a performance problem.

admin
├── dire
├── dira
│   └── a.log
├── dirb
│   └── b.log
├── dirc
│   └── c.log
├── dirf
└── dirs
    └── private.log

In the current case, in order to avoid such problems, we will generally specify specific subdirectories were collected configuration by creating multiple, very troublesome. In order to support such a scenario, a new targeting logtail blacklist acquisition path may be, of the specified directory, the file in the root directory is filtered based on the collection configuration specified by the skip logtail acquisition thereof.

Dependence logtail Version: 0.16.26 and above.

Function Description and examples

Currently, support blacklist function according to the directory path and file paths be filtered in two ways, the former will skip all the contents of the specified directory (files, subdirectories) acquisition, the acquisition of the latter is to skip the specified file. In order to increase the flexibility of use, when the path specified directory, file, can be used *, ?such as wildcards to match multiple targets .

We assume that the contents of the directory are collected /home/adminunder the *.logfollowing are some examples of blacklists and their effects.

• By directory path Ignore /home/admin/dir1: ignore all the content of subdirectories dir1.
• By directory path Ignore /home/admin/dir*: Ignore /home/admin/subdirectories dir beginning of all content.
• By directory path Ignore /home/admin/*/dir: Ignore /home/admin/all two subdirectories named dir content, such /home/admin/a/dirwill be ignored, and /home/admin/a/b/dirwill be collected.
• By file path Ignore /home/admin/private*.log: Ignore /home/admin/all directory privatebeginning, .logfiles ending.
• By file path Ignore /home/admin/private*/*_inner.log: Ignore the /home/admin/directory privatewithin the subdirectory beginning to _inner.logend of file, such /home/admin/private/app_inner.logwill be ignored, and /home/admin/private/app.logwill be collected.

Console operator

Text log [] by way of access (such as the JSON, separators, one-way, etc.), after entering the configuration wizard page. [Settings] Enable acquisition blacklist, you can use this feature to add a blacklist entry configuration as needed. The matching process will blacklist computational overhead, therefore, recommended that the number of blacklist entries not more than 10.