A. What ELK yes?
ELK is a complete log collection and display of elastic solutions provided by the company, it is the first letter of the abbreviation of three products, namely ElasticSearch, Logstash and Kibana.
ElasticSearch referred to as the ES, it is a distributed real-time search and analysis engine that can be used for full-text search, structured search and analysis. It is an established search engine Apache Lucene full-text search engine based on the use of the Java language.
Logstash is real-time data collection engine having a transmission capacity for data collection (eg: reading a text file), parses and sends data to the ES.
Kibana provides analysis and visualization platform for Web Elasticsearch. It can be found in Elasticsearch index, interactive data, and generate a variety of dimension tables, graphs.
II: ELK use of
the traditional sense, ELK is an open source alternative solution of Splunk. Splunk is the leader in log analysis. It includes not only log analysis system generates an error log, disorder, including the business logic, or any text analysis classes. And based on log analysis, it can produce a lot of solutions on it, such as:
Troubleshooting. We often say, operation and maintenance and development of this life is nothing more than a problem in and fight, so this sounds a simple words, in fact, is heavy. In fact, many companies are not short of money, we must stabilize, but to stability, development and operation and maintenance will be able to quickly locate the problem, even as a preventive measure, the problem is to kill in the cradle. Log analysis is clearly the cornerstone of technical troubleshooting. Log on to do troubleshooting, as well as a handsome technology, called full track link, such as Ali eagleeye or Google's dapper, it can be considered in a log analysis techniques.
Monitoring and early warning. Logging, monitoring, early warning are complementary. Log-based monitoring, early warning so that the operation and maintenance of mechanical clan has its own, significant savings in manpower and extend the operation and maintenance of life.
Related events. Log multiple sources of data generated by linkage analysis, through some analysis algorithms can solve various problems in life. For example, where the financial risk fraud. This can be applied to many areas, and depends on your imagination.
data analysis. For this data analysts, and algorithm engineers it is somewhat helpful.
Three: ELK reason fiery?
Before ELK, there is no similar solution?
A big God said yes, then it should be based on the Sphinx + Google char. Sphinx correspondence ES, Google char correspond Kibana.
That was why it did not fire, and now the ELK fire it? A more fantasy explanation is:
In fact the open source community will always have a variety of options, such as the lucene es java-based, there are based lucy c of the dezi. But who does not fire the fire who really is a very mysterious thing
I think the reason there are many ways, a more simple and central factor is the timing of the so-called hero is also. Of course, any one thing can not be a factor to cause, or we say that timing is a more general abstract factors.
Here I go from a number of factors explained why the ELK suddenly flourish?
1, data (logs) is increasing:
Early able to generate enough data to so some sites, and now a start-up company may need to face a massive user / mass request / analysis of massive pressure, which logs produced naturally very impressive, and as the business more complex, micro-service again taken seriously, regardless of system logs, traffic logs are still further. Operation and maintenance or development have found that I am from such a large system (hundreds of thousands of service) produced so many logs (100 billion), to troubleshoot problems, it is simply not possible. Used to have such a large amount of data companies are strong companies, they may have to deal with internal proprietary systems. But now suddenly become a popular demand, this time ELK advantage of the opportunity, will fall into place.
2, open source:
Open source has now been integrated into the IT community in the blood. Although we say commercial, self-development, as well as between the three open source are complementary, with profound feelings of love or occasionally kill, but if open source can be selected, apparently most of the development or operation and maintenance or preferred open source. Daniel said there was good:
Open source and convenience benefits of open source learning of the cost, will be able to recruit a person to develop his open dry, internal maintenance of a system, newcomers will always get angry a lot of problems; for example, to reconstruct might as well rewrite, or do not want to develop code on the basis of people's
ELK its open source attributes, apparently slightly better than Splunk.
3, attributes of the industry:
Some industries rely on the log is very large, such as CDN error logs in addition to the investigation, but also have an impact on many aspects of its CDN scheduling analysis, these are the real deal for money.
4, operation and maintenance of their own development:
Operation and maintenance is also developing itself, it could not have been years in the slash and burn. The log for the operation and maintenance, it should be regarded as the lifeblood. Into one system, the need for a standardized log analysis program, and it is a historical necessity. ELK at the right time production, operation and maintenance to accept he is a natural.
5, ELK own property issues:
Great God quoted a saying:
ELK able to address the core issues, coverage is broad, standardized, scalable integration, development, and operation and maintenance are its cold
ELK itself is very easy to use, it also has a very nice community, with demand so great that no fire will not work.
6, large data causing the machine started to become cheaper:
A good side effect of big data is to make a machine in a sense become cheap. Ranging from dozens of hundreds, many thousands or even hundreds of thousands. The rapid rise in the number of servers a lot to promote the development of technology, a typical example, now do not fire the depth of learning. This means that, out of dozens, hundreds of servers do log analysis, no problem, centralized log analysis slowly becoming mainstream. The ELK is a typical centralized log analysis program.
7, when writing computing -> calculate query:
The so-called calculation means writing data through a more complex processing, polymerization, directly to the query result obtained. Calculation rules determined by the demand for writing queries. With the development of storage formats, such as the popularity of columnar storage, etc., as well as powerful computing resources (ES a cluster moving the hundreds), so that data stored directly authentic, then the query time to do various calculations change It was possible. The ELK has provided a more powerful queries. Overall, the general direction is calculated at a conversion is written to the query. Calculate the maximum advantage when the query is to support any inquiry, no loss of information.
----------------
Disclaimer: This article is the original article CSDN bloggers' cup of rum ", and follow CC 4.0 BY-SA copyright agreement, reproduced, please attach the original source and link this statement.
Original link: https: //blog.csdn.net/u012562943/article/details/99946609