Recently, the selection results of the 2023 big data "Galaxy" case collection activity jointly organized by the China Academy of Information and Communications Technology and the China Communications Standards Association Big Data Technology Standards Promotion Committee (CCSA TC601) have been announced. Since the case collection was launched in September, it has received widespread attention and enthusiastic registration from the industry. After formal review and expert review, the "One-stop Traffic Big Data Platform Based on Hyper-Converged Database" jointly applied by Matrix Origin and Shenzhen Smart City Big Data Center Co., Ltd. was successfully selected as an outstanding database case among 706 application cases .
Case introduction
1. Case name
One-stop traffic big data platform based on hyper-converged database
2.Project background
Smart transportation applications face complex data processing requirements, that is, a large amount of heterogeneous data will be generated from various aspects such as people, vehicles, roads, and the environment. Traffic managers need to analyze and make decisions on these data in real time to respond to various traffic events. After undertaking a number of big data projects in the transportation field, Shenzhi City Big Data Company has formed a set of big data solutions based on traditional databases and Hadoop architecture. But in actual use, I found:
- There are major flaws in traffic scenarios : it cannot meet business needs such as ultra-large-scale high-frequency data writing, real-time data analysis, real-time data update, data consistency guarantee, and heterogeneous data compatibility.
- There are major problems in management complexity : Hadoop technology stack components are scattered and operation and maintenance are complex, which poses great challenges to rapid delivery and long-term operation and maintenance.
- The compatibility with emerging cloud-native infrastructure (such as K8s) is not high enough : the underlying hardware resources cannot be fully utilized, and management and maintenance are fragmented.
3. Project process
Matrix Origin took 10 months to help it migrate hundreds of TB of offline data and dozens of TB of real-time data from Hadoop and ClickHouse to MatrixOne, and carried out special performance optimization of real-time analysis links for business needs; and targeted Some semi-structured and unstructured data have customized data types and UDF functions. Ultimately, the success enabled Shenzhen Smart City Big Data Center Co., Ltd. to reduce the number of data components by 80% and reduce operation and maintenance costs by 50%. This case is the first big data business project in the transportation industry to adopt a hyper-converged database. It is also the first project to natively use K8s to unify and fully containerize the data layer and business layer.
Case value
The successful selection of the big data "Galaxy" case proves that the Matrix Origin team has demonstrated excellent professional skills and technological innovation capabilities in the project , bringing significant benefits to Shenzhen Smart City Big Data Center Co., Ltd., and also It provides a new data processing idea and solution for the entire transportation industry. This case has had a profound impact on the industry and contributed significantly to the widespread application of big data technology in the transportation industry and the advancement of the industry.
About the origin of the matrix
Matrix Origin is an industry-leading big data and database management system (DBMS) technology and service provider. Its main team members come from well-known domestic and foreign technology companies and have strong innovation capabilities. Matrix Origin's goal is to create and use world-class data infrastructure technologies and products to assist enterprises in transforming and upgrading from informatization, digitalization to intelligence. Matrix Origin has core competitiveness in the fields related to cloud computing, databases, big data and artificial intelligence. It has a broad industry and international vision and foresight, and can quickly and effectively implement advanced technologies in different fields and expand them on a large scale.
About MatrixOne
MatrixOne's core product, MatrixOne, is a multi-mode database based on cloud native technology that can be deployed in both public and private clouds. This product uses an original technical architecture that separates storage and computing, separation of reading and writing, and separation of hot and cold. It can simultaneously support multiple loads such as transaction, analysis, flow, timing, and vector under a set of storage and computing systems, and can perform real-time and on-demand Isolated or shared storage and computing resources. MatrixOne can help users significantly simplify the increasingly complex IT architecture and provide minimalist, extremely flexible, cost-effective and high-performance data services.
About Shenzhen Smart City Big Data Center Co., Ltd.
Shenzhen Smart City Big Data Center Co., Ltd. is a wholly-owned subsidiary of Shenzhen Smart City Technology Development Group Co., Ltd. It is committed to providing high value-added information systems and services based on high technology and high quality, and becoming a customer-friendly enterprise. Solution services enterprises that contribute to value creation.
The current main business of big data companies is the construction and operation of big data centers, state-owned and state-owned enterprise cloud and development of Kunpeng industry, and promotion of data legislation.
Big data companies are committed to providing high value-added information systems and services based on high technology and high quality, becoming a solution service company that can contribute to customers' corporate value creation, and creating ecological clusters to become new urban smart platforms, gathering The resources of all parties realize the development of the entire industry chain and become a customized multi-cloud management service provider.
MatrixOrigin official website: A new generation of hyper-converged heterogeneous open source database-MatrixOrigin (Shenzhen) Information Technology Co., Ltd. MatrixOne
Github 仓库：GitHub - matrixorigin/matrixone: Hyperconverged cloud-edge native database
Keywords: hyper-converged database, multi-mode database, cloud native database, domestic database.
Bilibili crashed twice, Tencent’s “3.29” first-level accident... Taking stock of the top ten downtime accidents in 2023 Vue 3.4 “Slam Dunk” released MySQL 5.7, Moqu, Li Tiaotiao… Taking stock of the “stop” in 2023 More” (open source) projects and websites look back on the IDE of 30 years ago: only TUI, bright background color... Vim 9.1 is released, dedicated to Bram Moolenaar, the father of Redis, "Rapid Review" LLM Programming: Omniscient and Omnipotent&& Stupid "Post-Open Source "The era has come: the license has expired and cannot serve the general public. China Unicom Broadband suddenly limited the upload speed, and a large number of users complained. Windows executives promised improvements: Make the Start Menu great again. Niklaus Wirth, the father of Pascal, passed away.