Grandchildren, Hadoop has been born for 13 years. It was first born in 2006, and became the top-level Apache project in 2008. Not long after the birth of the Internet has become a big data computing industry standard configuration, but also became one gold medal in the project of the Apache Software Foundation. However, beginning in 2016, it began to appear at home and abroad Hadoop bad-mouthing the voice of the future represented by Hadoop open source big data go from here?
Want to be a big data cloud computing Spark master, look here! I read poke
50W annual salary of Java programmers to turn Big Data learning route poke me read
Artificial intelligence, big data trends and prospects poke me read
The latest and most complete big data exchange system path! ! Poke me read
2019 latest! Big Data Engineer jobs salary, it was amazing ! I read poke
Ten years ago, three well-funded start-up company Cloudera, Hortonworks and MapR begin to commercialize around the open-source Hadoop ecosystem products and services. Speculation about Hadoop in early 2014 reached a peak. At that time, Cloudrea to raise a huge amount of approximately $ 900 million of financing, valued at $ 4.1 billion.
"The recent dispute Cloudrea and MapR on a lot of headlines, so that the public could not help wondering what the dispute mean for the future of big data." Kunal Aganwal Unravel Data CEO. "Corporate interest in the data that is weakening it? Did not matter. Just because of the large data quickly transition to a public cloud, causing these companies faltered, those designed for local deployment designed platform has no growth potential of. Due to the large data for the calculation of highly elastic demand, it is more natural for cloud computing. in addition, modern data systems are becoming increasingly complex, they are more difficult to manage than local management in the cloud. with the new data stack turned out, Hadoop big data is no longer the authority of the technology: like Spark and Kafka such techniques are emerging to support modern data applications .Hadoop use of artificial intelligence and machine learning will not disappear, not all data workloads are migrated to the cloud but public cloud and Spark technology will increasingly define big data, do not actively support any of their suppliers will continue to suffer losses. "
Hortonworks market in 2014, Cloudera followed, listed in 2017. But as the market competition intensifies, customers began to quickly turn the cloud, the two companies share prices have plummeted. Last fall, Cloudrea and HortonWorks merger, the combined company's stock continued to fall, the market value has shrunk by half. MapR in more than four years ago, announced the listing plan, but never implemented, but chose to re-raise two venture capital in 2016 and 2017. The recent news that, if MapR can not get more money, may cut up to 122 jobs and close headquartered in Santa Clara, Calif.
"Recent Cloudrea and MapR news triggered a debate about the future of Hadoop go from here, and all open-source framework for managing big data workloads." Chandra Ambadipudi commented CEO Clairvoiant company. "An important factor is, Hadoop in the management and utilization of the resources it needs to be aspects of the market greatly underestimated .Hadoop did make it as a low-cost, scalable and robust commitment to open source solutions. But the management complexity the number of personnel and engineers required data, as well as their shortage, have reached its peak. "
Now, Cloudrea become the only important Hadoop company, after a storm MapR news, here are some insights and ideas about the future of open-source Big Data platform (computing giant and Microsoft, AWS, Google and other cloud) from the local to the cloud.
-
Hadoop suffered questioned the feasibility of, not because it is a cake trough technology (in fact, Hadoop technology is very good), but because the Hadoop open source products to manage as too complex to be faced with personnel bottlenecks. Compared with the hype, the required level of resources has been greatly underestimated.
-
The question is whether cloud computing giant will completely occupy the field? Databricks and Snowflake are addressing skill gaps and implement aspects of big data.
-
Ecosystem still appear integrate behavior (like Microsoft acquisition MapR the same), only time will tell whether all this is good for the ecosystem (vendor lock-in).
-
Similarly, other emerging big data platforms such as Apache Kafka, may also face the challenge of open source solutions (like Cloudrea the same challenge as publishers open source Hadoop's face).
"As cloud computing giants continue to 'devour the world', like Snowflake and DataBricks such a platform is also rising, began to try to bridge the gap big data talent and skills," Ambadipudi added, "if further consolidation in the market, such as there is a some cloud computing company acquired MapR Hadoop and other companies, I would not be surprised. Because of low latency and scalability, Kafka increasingly popular, and has been widely adopted. but like Cloudera Hadoop use the same, Confluent the Kafka enterprise Edition is also doing the same thing, so they could face the same challenges with open-source platform. No matter what kind of data to achieve big way, the current skills shortages are required, and the demand for expert management services will remain high. "