On June 16, the "Big Data and Microservice Technology Innovation Forum for Fast-Growing Enterprises" jointly sponsored by Huawei Cloud, msup, Xiamen Torch University, and Xiamen Industry Software Association was successfully held in Xiamen. This event brought together CTOs and technical experts from well-known companies such as Huawei Cloud and Zhenai.com, analyzed the construction of big data platforms and the evolution of microservices through technical cases, and discussed in depth how to develop and manage data around the data platform. Helping architects, technical managers, etc. find the best practice for building a lightweight big data platform, and quickly mine value from data to feed back business.
Event site
At the beginning of the event
Huawei Cloud Fujian CTO
He said that HUAWEI CLOUD has developed rapidly in recent years. The emergence of HUAWEI CLOUD has provided enterprise customers with a good choice, made the development of the cloud computing market more standardized and healthy, and made cloud vendors have more respect for customers and respect for customers. Data sovereignty is also maintained with more respect. "Serve customers well and enable customers to use the cloud well" is the purpose of HUAWEI CLOUD.
In Fujian, leading customers in various market segments have chosen HUAWEI CLOUD, such as Meitu in the Internet industry, Shenzhou Bicycle and Ketuo in the upstream and downstream industries of automobiles, Industrial Bank in the financial and insurance industry, and Anta, 361, Jiu Mu and others. HUAWEI CLOUD is growing with more users.
HUAWEI CLOUD Big Data Technical Expert
Next, HUAWEI CLOUD big data technology experts published a sharing titled "Building a Modern Data Stack to Accelerate the Release of Data Value".
In the era of digital economy, data has become an important means of production, stimulating the development of digital economy. According to authoritative data, the opportunities for industrial digitalization in 2025 will reach 23 trillion US dollars. The digital transformation of enterprises is being upgraded, and data-driven business intelligence is increasing day by day. In this process, enterprises are faced with the problems of high technical threshold, high investment cost, and difficult data governance. To address these pain points, HUAWEI CLOUD provides customers with an easy-to-use full-stack big data infrastructure.
The storage-computing separation architecture of HUAWEI CLOUD's big data infrastructure has further evolved. DataArts, an intelligent data production line, can help enterprise data enter the lake, analyze, and process real-time data. Its AI capabilities can realize intelligent data preparation and governance. DataArts also has full-link data security management, which can protect private data and conduct compliance audits on data usage. The HUAWEI CLOUD big data infrastructure can also freely flow between big data and AI engines to achieve digital intelligence integration. DataArts Studio, Huawei Cloud Data Governance Center, is a one-stop end-to-end platform from data integration to development, governance, and service. In terms of full-scenario data integration, full data, incremental data, and real-time data can be integrated with one click. In terms of data development, through one-stop development + online + operation and maintenance, unified development environment to improve data development efficiency, and a low-code development platform can greatly improve the development efficiency of developers.
Finally, HUAWEI CLOUD big data technical experts introduced in detail the practical cases of HUAWEI CLOUD empowering Mengbao Group to operate intelligently, make decision-making intelligent, and help Mengbao develop new value for e-commerce.
Zhang Jing, senior technical director of bilibili
Data governance refers to the management of the availability, usability, integrity, and security of data used in an organization. A data governance platform is a tool that helps organizations effectively manage their data governance processes. Zhang Jing, senior technical director of bilibili, introduced the big data development and governance platform driven by data governance in detail from the concept of big data governance platform, the problems of platform construction in station B, and how to use data-driven methodology to promote the direction of platform construction. building.
Interpreting the big data development and management platform from a global perspective can be divided into two modules, three scenarios and four types of users. The two major modules include development and governance modules. Development refers to the scheduling system and code IDE required for daily development. Governance includes data management as well as efficiency and governance tools related to security, cost, and quality. The three scenarios include data production scenarios, data consumption scenarios, and data governance scenarios. The four types of users include data analysts, data RDs, algorithm RDs, and data operations. Station B conducted a simple survey of third-party solutions on the market. Out of cost control, integration capabilities, and performance requirements, Station B finally chose a self-developed solution.
Next, Zhang Jing introduced the business product architecture of station B, and the pain points encountered in the five-year platform construction, and focused on how to use the methodology of data governance to drive platform construction. He said that the data governance platform needs to be closely connected with the business, and should prioritize solving business problems. To deal with some new technologies, it is necessary to conduct research and decision-making in terms of functional process, scenario benefits, and DEMO links. In terms of organizational form, there must be a self-sustaining business. If there are too many requirements, try to make an assessment according to the priority of quality>cost>safety. Of course, the platform must operate data instead of just being a tool.
Huawei Cloud Technical Expert
With the development of cloud-native technologies such as microservices, serverless, DevOps, and low-code development, the earliest enterprises that started to practice application modernization have also reaped benefits such as improved customer experience. Therefore, the time for comprehensive application modernization is ripe, and application modernization has It is imperative. So how can enterprises quickly realize the modernization of applications? Huawei cloud technology experts said that microservices and serverless are important technologies. Through these two technologies, a single application can be upgraded to an elastic and scalable distributed application, and bring about delivery methods, R&D models, and security on the cloud. innovation.
HUAWEI CLOUD's application modernization, based on technologies such as cloud native, AI, and serverless, helps enterprises upgrade traditional applications through relocation, transformation, and reconstruction, and create new applications with ultimate experience, intelligence, efficiency, security and reliability, and achieve business agility and efficient innovation, unleash the digital productivity of the enterprise, and achieve a modern enterprise with high-quality growth. Among them, ServiceStage is a one-stop PaaS platform service of Huawei Cloud, which provides application cloud access, application operation and maintenance, observable integration capabilities, application release function, continuous deployment support, and multiple release strategies (rolling, canary, blue-green) , to ensure the stability of changes.
In addition to the ServiceStage service, HUAWEI CLOUD also provides a fully managed, serverless PaaS platform service CAE, which further reduces the threshold for applications to go to the cloud, improves O&M efficiency, and improves resource utilization. At present, HUAWEI CLOUD microservices have been widely adopted by large enterprises, central state-owned enterprises, ministries and commissions, and government affairs.
Peng Wanliang, CTO of Zhenai.com
Zhenai.com is a dating website with a history of 17 years. It has experienced explosive growth in business scale, system bottlenecks, business development history such as micro-service splitting, link optimization, cloud migration, containerization, and dual-cloud active-active. , and finally precipitated a set of microservice governance architecture with precious characteristics. Peng Wanliang, CTO of Zhenai.com, explained the architecture in detail in his speech.
During the architecture development process, Zhenai.com encountered some pain points, such as: insufficient resource isolation, difficulties in parallel requirements joint debugging/testing, too much human participation in the deployment process, and difficult management of multiple environments. In order to solve these problems, Zhenai.com introduced docker and k8s to replace the original deployment and management methods of jar packages and shell commands; transformed the development and testing environment to solve the problems of parallel concurrency and difficult testing; transformed the release process, solidified the release process, simplified operations, and reduced Error; service unitization, refine resource isolation granularity.
The dual-cloud active-active architecture greatly avoids the risks of cloud platform downtime and data loss caused by the failure of a single cloud environment, and can optimize enterprise workloads according to needs, better leverage the elastic advantages of the cloud, and realize resource reuse, The goal of dynamic adjustment, real-time verification, and fast switching.
Finally, Peng Wanliang summarized the core of the evolution of Zhenai.com’s micro-service architecture as follows: continuous business development, continuous iteration of the architecture to expose design defects, or better solutions for the business, and then focus on various problems, and finally rise to the adjustment of the framework level .
The event came to an end amid heated discussions and unfinished business. In the future, HUAWEI CLOUD will unite with more ecological partners to jointly release more and better solutions, help partners sort out the strategic map, and jointly create more possibilities for cooperation.