Topic: Practice and Exploration of Bank's Distributed Database Reconstruction
Outline:
1. What are the advantages or disadvantages that a distributed database can bring to us?
2. Which development skills are recommended for database practitioners
3. For new DB-related practitioners, share experience and suggestions
4. Distributed databases are related to the calculations in the table below Scenarios, recommend how to control
5. What are the points to pay attention to when migrating from traditional data tables to distributed databases
I heard that 3306π is going to do something
Yes, 3306π Shenzhen station is about to be done soon
Ah, is it? What day, where and how to register?
On December 19th, in the coastal city of Nanshan District , Shenzhen , you can register by clicking the original link at the end of the article or scanning the QR code. Let's brag together.
Guests introduce themselves
Wang Hui is a database expert with 15 years of experience in Internet and financial industry architecture and IT operation and maintenance management. He focuses on the field of data and database. He is good at building system security and intelligent operation and maintenance systems. He is good at system architecture optimization and full-link analysis and diagnosis; rich IT projects Experience in development, design and management; proficient in relational database MySQL/Oracle, non-relational database Redis/MongoDB, in recent years, mainly engaged in the research and promotion of domestic and open source distributed databases, and the transformation and migration of heterogeneous databases .
01
May I ask Mr. Wang, what advantages or disadvantages can distributed databases bring us?
Advantage:
1) High scalability: When business data continues to expand and performance becomes bottlenecked and cannot be improved by increasing hardware resources, distributed databases can be resolved by splitting data and expanding horizontally
2) High reliability: Distributed database has the characteristics of multiple copies, and the data consistency of the copies can be ensured based on the consensus protocol
3) High availability: It can automatically switch over node failures without manual intervention and is transparent to applications
insufficient:
1) Since the distributed database links the data nodes together through the network, the network bandwidth requirements are high, and any problems caused by network delays will be magnified
2) Most of the distributed transactions use the 2PC method. Although the products of various manufacturers have been optimized, but subject to the principle, there will still be a certain loss in performance
3) There are restrictions on specific objects and grammar, and there is a certain transformation cost for development
4) Relatively low maturity, high learning and operation and maintenance costs
02
Teacher Wang has rich experience. As a database practitioner, development skills are essential. Which development skills do you recommend?
The basic Shell scripting language can realize simple automatic operation and maintenance; Phyon language can realize more complex automation, even the requirements of intelligent operation and maintenance management, it is very recommended to master and use it.
03
As a successful person, Mr. Wang, can you share your experience and suggestions for new DB-related practitioners?
1) Cultivate your own good operating habits, be in awe, and don't operate blindly based on your feelings.
2) Cultivate your own thinking about troubleshooting. For production failures, don’t think too much about it. You can quickly deal with it by asking others for the first time, and then slowly analyze and research, and put aside the rapid troubleshooting In the first place.
3) Cultivate your own habit of recording problems, and improve your writing and problem analysis skills through continuous accumulation.
4) Cultivate your ability to continuously learn and innovate, and slowly realize how to do your work better, more efficiently and simply.
04
How do you recommend controlling the computing scenarios associated with the table in the distributed database?
1) Table association is generally not recommended for links with more than 3 tables. It needs to combine business scenarios and business logic. For distributed database table associations, shard keys must be used to ensure that data is associated with each shard node. Broken shard nodes to avoid the data aggregation overhead of broken nodes caused by network broadcasting;
2) For the association of non-sharded tables, the problem of node collapse can be solved by establishing a global table;
3) It is necessary to cultivate the ability level of developers, system-related development specifications, establish SQL audit procedures, and try to avoid the phenomenon of simple business logic and complex realization.
05
May I ask Mr. Wang, are there any points that need to be paid attention to when migrating from a traditional data table to a distributed database?
1) The object and syntax compatibility issues of the database determine the development and transformation costs
2) The performance problem after migration, the performance should not be lower than the original traditional database
3) Data consistency issues after migration should be verified repeatedly to ensure that the data is consistent before and after migration
4) Migration time window and operational risks, according to different time windows and data volume system different migration plans, while avoiding man-made operational risks
5) Comprehensive business testing: including application function, application performance, stability and other tests. The data is for the service business, and the final success depends on the business. Therefore, in the testing phase, a comprehensive business verification must be done on the migrated data test
6) Making a detailed migration plan and finding a perfect migration tool will do more with less
7) System rollback strategy to deal with problems that can only be solved by rollback in extreme situations.
In general, the principle of data integrity and consistency, the principle of stable business operation, the principle of controllable costs, and the principle of trustworthy rollback should be achieved.
Scan code to sign up
You can also register by clicking the " Read the original " link at the end of the article
participate
Learn new skills
Win Luo Xiaobo's new book "MySQL Replication Technology and Production Practice" and the Aikesheng exquisite cup
rule of activity
In "Young people do not speak martial arts? On December 19th, come to 3306π Shenzhen Station to learn a little bit of real skills" leave a message below
Write down your suggestions for this event
Or a question you want to ask the guest
After a valid message is selected, the number of likes exceeds 30
Take the top 10 according to the number of likes
give presents
Choose one of the two
About the "3306π" community
Focusing on the core technology of MySQL, bring the most important data solutions in the Internet industry to traditional industries; include other open source technologies RadonDB, ClickHouse, Redis, MongoDB, Hbase, Hadoop, ElasticSearch, Storm, Spark, etc.; share dry knowledge, even if It’s a sponsor, and asks for it, refuses to release