On November 11, the MatrixOne community successfully held the second MatrixOne Meetup in Shenzhen . On the day of the event, dozens of external partners attended and shared knowledge about cloud native databases. In this event, we also invited external lecturers from Shenzhen Suwen Intelligence to share content related to the currently popular large model industry. Through lively discussions, exchanges, and interactions, participants and MatrixOne deepened their understanding of each other. We will hold the third MatrixOne Meetup in Beijing, where we will invite customers to share relevant cases; in the future, MatrixOne Meetup will also be held in multiple cities, and everyone is welcome to continue to pay attention and participate!
Let’s review this event together~
At the beginning of the event, Deng Nan, product director of Matrix Origin, explained the new generation of data trends in the cloud native era . Deng Nan pointed out that at the application layer, the current unified platform of K8S and containers has become an inevitable trend. In the data layer, K8S+S3+Serverless is becoming a new paradigm of data architecture. Data processing requirements are also developing towards unified integration and multi-mode. Large model tool chains and data processing capabilities are also more closely integrated. Under the new technology development trend, what kind of database do users need? In Deng Nan's view, what users currently need is a cloud-native serverless, hyper-converged All-in-One database that can carry out minimalist data development, and MatrixOne is just such a product that adapts to the development of the times and meets user needs.
Subsequently, Wu Yelei, senior cloud native R&D engineer of Matrix Origin Products, explained how MatrixOne builds an HTAP database based on K8S and S3, explaining how MatrixOne achieves serverless and hyper-convergence capabilities. From the perspective of a technician, he comprehensively explained the design concepts of MatrixOne cloud computing resources, storage systems, and each layer of architecture . It expresses that moving databases to the cloud is no longer new, but fully realizing the potential of the cloud requires cloud-oriented design. The unified API of K8S makes the cloud ubiquitous. Currently, cloud-native applications have matured, and data systems are no longer an exception.
After a short tea break, Matrix Origin Product Manager Li Song shared MatrixOne application scenarios and demo demonstrations, showing the current research and development results of MatrixOne and MatrixOne Cloud. ; Li Song showed the MatrixOne application demo and elastic expansion and contraction-related demos in the Internet of Vehicles scenario. MatrixOne Cloud has extremely high cost performance and resource isolation capabilities based on SQL billing, and realizes flexible expansion based on resource isolation. Li Song pointed out that MatrixOne can effectively optimize users' pain points in using databases in HTAP, SaaS and time series + real-time data warehouse scenarios, bring users the ultimate experience of high performance and cost-effectiveness, and help users reduce costs and increase efficiency.
Finally, this Meetup invited Wang Wei, the founder of Shenzhen Suwen Intelligence , to tell the story step by step and from simple to profound with four interesting stories: Turkey, Dartmouth, Big Prophecy Model and Industry Chain + Big Model. Current large model applications in industrial big data.
In addition to the speech session, as usual, this Meetup also prepared exquisite tea breaks and rich gifts for the users present. During the meeting and coffee break, we collected and compiled the wonderful questions raised by the students present:
Q: Does MatrixOne Cloud meet industry security standards? How is the security isolation of the product implemented?
A: MatrixOne Cloud is designed to meet industry security standards, and various third-party certifications are in progress. In terms of security isolation, MO Cloud uses the cloud provider's storage encryption to ensure data encryption at rest, and is considering supporting different tenants to use their own keys to encrypt data in the future to provide higher security isolation.
Q: If a transaction is started during a large amount of writing, will it cause a lot of intermediate states, cause conflicts, and lead to being overwhelmed and other users unable to write?
A: MatrixOne does Snapshot Read. For each transaction, CN only needs to wait for the data timestamp in memory > transaction timestamp to start processing the transaction. The intermediate states are all in the local workspace of CN, and finally commits to TN. It will not Overwhelm TN or LogService.
Q: Will Serverless automatic expansion have a CN cold start state?
A: Yes, each new CN needs to load the object storage data and LogTail data to be read by SQL into the memory before calculation can be performed. The amount of LogTail data is small, and the cold start time mainly depends on how much S3 data needs to be read. The less data to be read (for example, most of the data hits the CN's Local Cache), the shorter the cold start time will be.
Q: Are there any overflow alarms for high-concurrency filing?
A: The instantaneous demand for a large number of new computing resources is difficult for private clouds to meet immediately, but it can be supplemented by public cloud resources. This is also one of the scenarios of cloud-edge collaboration. We also provide overflow alerts.
Q: Is there a big difference between the community version and the cloud version? Can the community version store hot data locally and cold data in S3?
A: The community version does not have the ability of Serverless cloud platform to automatically control CN resources. You need to specify how many and how large CNs to deploy. The community version can put hot data locally and cold data into S3, as long as there is a connection between the local and S3 or other object storage. With stable network communication, you can deploy MO locally and use object storage on the cloud at the same time.
Q: Will multi-tenant isolation limit CPU or storage usage on the cloud?
A: For computing resources, paid instances will use independent CN groups. A single CN instance in each CN group will use containers to limit CPU/Memory, and the upper limit of the CN group will be (nearly) infinitely expanded. There are no storage limits and you pay per space.
Q: Is multi-tenant a common computing node?
A: The policy is configurable. Taking MatrixOne Cloud as an example, different paid instances (even those of the same tenant) will use independent computing nodes (CN), and free instances will use a set of shared computing nodes (CN).
Q: Will one node mobilize multiple Pods?
A: The current strategy is to schedule one MatrixOne Pod per node.
Q: Can MatrixOne be read in real time? Is the CDC capable?
A: MatrixOne supports real-time data sharing between instances through the publish and subscribe function; however, it does not yet support synchronizing data to downstream through CDC.
Q: Should the existing query be expanded or the next query be expanded?
A: MatrixOne Cloud Serverless's current automatic expansion strategy is relatively conservative. The system will observe the resource usage of the current instance to determine the capacity expansion to meet subsequent query requirements.
Q: Will a CN access the Connections of multiple tenants?
A: Free instances on MatrixOne Cloud will share CN Pods, but paid instances will have an exclusive CN Group. The link of this paid instance will be evenly distributed to each CN Pod.
Q: Are the CPUs corresponding to a CN the same or different?
A: It is currently the same. The same specifications are easier to maintain and expand. But we are also planning CNs of different specifications for business use under different loads to achieve accuracy and smoothness.
Q: Are the number of CNs and performance linear? Are there any charges for the Internet?
A: It is difficult to achieve absolute linear expansion because the distributed system will have the task overhead of data exchange. However, judging from the current test results of MatrixOne, the linearity is still quite high; the network between CNs is not charged separately. , CU's billing will only consider object storage I/O requests.
Finally, I would like to thank all the guests and friends who attended, and look forward to seeing you again at MatrixOne Meetup Beijing!
About the origin of the matrix
Matrix Origin is an industry-leading big data and database management system (DBMS) technology and service provider. Its main team members come from well-known domestic and foreign technology companies and have strong innovation capabilities. Matrix Origin's goal is to create and use world-class data infrastructure technologies and products to assist enterprises in transforming and upgrading from informatization, digitalization to intelligence. Matrix Origin has core competitiveness in the fields related to cloud computing, databases, big data and artificial intelligence. It has a broad industry and international vision and foresight, and can quickly and effectively implement advanced technologies in different fields and expand them on a large scale.
About MatrixOne
MatrixOne's core product, MatrixOne, is a multi-mode database based on cloud native technology that can be deployed in both public and private clouds. This product uses an original technical architecture that separates storage and computing, separation of reading and writing, and separation of hot and cold. It can simultaneously support multiple loads such as transaction, analysis, flow, timing, and vector under a set of storage and computing systems, and can perform real-time and on-demand Isolated or shared storage and computing resources. MatrixOne can help users significantly simplify the increasingly complex IT architecture and provide minimalist, extremely flexible, cost-effective and high-performance data services.
MatrixOrigin official website: A new generation of hyper-converged heterogeneous open source database-MatrixOrigin (Shenzhen) Information Technology Co., Ltd. MatrixOne
Github 仓库：GitHub - matrixorigin/matrixone: Hyperconverged cloud-edge native database
Keywords: hyper-converged database, multi-mode database, cloud native database, domestic database.
Bilibili crashed twice, Tencent’s “3.29” first-level accident... Taking stock of the top ten downtime accidents in 2023 Vue 3.4 “Slam Dunk” released MySQL 5.7, Moqu, Li Tiaotiao… Taking stock of the “stop” in 2023 More” (open source) projects and websites look back on the IDE of 30 years ago: only TUI, bright background color... Vim 9.1 is released, dedicated to Bram Moolenaar, the father of Redis, "Rapid Review" LLM Programming: Omniscient and Omnipotent&& Stupid "Post-Open Source "The era has come: the license has expired and cannot serve the general public. China Unicom Broadband suddenly limited the upload speed, and a large number of users complained. Windows executives promised improvements: Make the Start Menu great again. Niklaus Wirth, the father of Pascal, passed away.