Recently, Shay Banon Elasticsearch founder and CEO sent a "special" message to Tencent cloud, made a special trip to Tencent team thanked for the contribution to the open source community Elasticsearch made.
It is understood that Tencent engineer by submitting codes successfully optimized Elasticsearch high concurrent write performance, achieving 10 million when the amount of data written to the long cut from the original 18-15 minutes, the rate increased by 20% and the open source community to share in Elasticsearch the project plan.
Shay Banon in distributed vice president of Tencent, Tencent cloud letter of thanks, President Qiuyue Peng said:
"We recently carried out a major Elasticsearch and clever improvement, which is proposed by a developer Tencent. This improvement allows certain types of write speeds Elasticsearch increased by about 20%, and we look forward to continue to follow Tencent cloud maintain such good relations of cooperation. "
In fact, Tencent global digital ecosystem of the General Assembly in May of this year, Tencent cloud and Elastic signed a strategic cooperation agreement. The two sides agreed in products, technology, ecology three dimensions of cooperation, relying on Elastic products and technology advantages in the field of search software, as well as technical and market resources Tencent cloud computing in the field of big data in the cloud, the two sides jointly launched with advanced business features Tencent cloud elasticsearch Service, the on-line version of the cluster rights management, SQL, machine learning, and other advanced features Figure inquiry. Tencent cloud combined with the world's leading public cloud infrastructure, low-cost Elasticsearch clustering applications, easy to develop, easy management, high availability, high security.
In fact, the opportunity to optimize source code, derived from the internal Tencent stop intelligent big data platform universal service BI real business scenarios, in which all the people of BI statistics, at the bottom of the Elasticsearch use of open source code. In actual use, since there is historical data than the demand, the need for the full amount of daily import new data. Number one billion of the amount of data, more than 500 labels, you need to quickly import as much as possible into the ES system, but this one after several parameter optimization, still take several hours to complete the import, the entire system is a big bottleneck.
In this context, BI Tencent national team began to optimize write performance under high concurrency Elasticsearch scene. Mechanism project team by writing to the ES, in multiple stack sampling analysis, we found many threads is because writeLock rollGeneration occupied, took rather long in the waiting to acquire the lock. Thus the optimized focus transferred from tuning parameters to tune thread. Long cut when the project team final on the basis of the theoretical value of the experiment, by optimizing rollGeneration, will achieve 10 million the amount of data written from the original 18-15 minutes, rates increase by 20%.
"Very grateful for the support and advice one Elastic founder and technical director Simon Willnauer of this optimization greatly enhance the efficiency of data analysis for All BI system, then these leading features are able to get on Tencent cloud Elasticsearch Service . "The main official said.
More to look forward to is that this optimization is only a beginning, there is still more room for optimization in the future, Tencent cloud is now actively communicating with the open source community.
It is worth noting that this is not the Tencent community code contributions of individual cases, starting in 2010, Tencent began the "three-step" open source projects, and actively promote the code open and community involvement, isolated from a closed to an open and collaborative the depth direction of development of community open governance.
"We will, by way of external open source and actively participate in building a global community of science and technology. " Chairman of the Board and CEO of Tencent Ma said at the 2019 Summit Tomohiro will be held in Chongqing in late August.
Today the company being in a different way sponsors, contributors and other actively involved in open source. In terms of contribution to the community, Tencent has received multiple Platinum or Platinum sponsorship OpenStack community, Linux Foundation, LFAI Foundation, MariaDB Foundation, Apache Foundation.
Last year, Tencent cloud contribution to the KVM kernel 40 patch, boarded the second year in a row KVM open source contributions to the global list, China has become the only selected cloud vendors. Earlier this year, Tencent officially released cloud infrastructure level four core technology projects, covering respectively data centers, networks, servers, and field automation platform, and announced that these four technology projects all contribute to the open source community OCP.
In the autonomous sources of income, as of August 2019, Tencent has been published in open source projects on Github 82, contains micro-channel, cloud Tencent, Tencent games, Tencent AI, Tencent security and other related fields. Tencent launched an open source project on Github won a total of 24W + Star count in the domestic and international gains widespread attention and recognition.
Future, Tencent will further broaden the scope of open source, open, shared jointly develop a research and development mode, and constantly reinforce the existing technological capabilities, and constantly deepen the benign relationship with open source partners mutually reinforcing.
