Chapter IV high-performance memory
relational database
Separate read and write (access relieve stress)
Rationale: The database dispersed read and write operations on a different node, a pressure reducing access a single database, improve access efficiency.
Basically:
- Build a database server from the cluster master, a master-slave or a master multi-slave.
- Host is responsible for the database read and write operations, the machine is responsible for the read operation.
- Host database synchronized by copying the data to the slave.
- The service server transmits to the host write, the read is sent to the slave.
Affairs issues: consistency.
【problem】
How to ensure data consistency of master and slave? ? ? The main problem of delayed copied from.
- Second reading, reading from the host machine to read again
- Point to a host of business-critical, non-critical business point Slave
Sub-library sub-table (to reduce the storage pressure)
Sub-libraries
The service module into different database server in. Business users such as electric projects, goods, orders can be on a different case three servers.
【problem】
join operational problems
Can not be achieved related inquiries
Affairs issues
Data need to be consistent. For example, the order plus a number of commodities will be reduced by one. (Latency issues)
cost
Points table
Single table data split Split horizontal and vertical split two kinds.
After the split can be placed in the same database, it can be placed in different databases.
Vertical Table
The table columns are not used to split out. Increase the number of tables will bring complexity. But can significantly improve query performance.
The level of sub-table
The level of sub-table for a particularly large number of rows in the table table.
The level of problem points table
Route: Depending on what condition split table
Route scope: The data column as the ordered routing chromatographic conditions, such 1-999999,1000000-1999999.
Recommended segment size between 1 million to 20 million
Disadvantages: selecting segment size complexity; but can be extended with the increase in the new table data smoothing
Hash routing
Configuring Routing
join operation requires the merged results
order by operation can not be performed in the database, only query the database through code or middleware business separately, and then aggregated sorting
NoSQL
NoSQL Category:
- KV storage: Redis
- Document database: MongoDB
- Columnar database: HBase
- Full-text search engine: ElasticSearch
Cache
Rationale: The data may be reused in memory, generated once, use many times.
[A star send a micro-blog, performing one insert, and a plurality of select n]
Cache penetration
Cache does not play a role, business systems read into the cache, but no data.
problem:
- Stored data does not exist
- Cache takes a long time
Cache avalanche
When the cache invalidation sharp decline in system performance. Many requests to access the database, while generating the cache.
solution:
- Update lock, only one thread is generated cache. (DLM)
- Background updates. Thread updates do business, but specifically updated with the background thread.
Cache hotspot
Multiple copies of the cache, the cache server cluster to create, distribute requests to different servers.
[Xinlang such fans over the star 100w made to it, and generates the cache 100 parts (of course, requires the server 100)]
Chapter V high-performance computing
From the physical level is:
- Try to improve the performance of a single server, will play resources to the limit
- Single server to achieve performance bottlenecks, server clustering scheme design
High-performance single-server
High Performance Cluster
Instead of the previous load balancing task allocation.
Load balancing Category
- DNS load balancing (geographic level to achieve load balancing)
- Hardware Load Balancing (similar routing switch) or more concurrent 100w is expensive, it is a good a BMW.
- Software load balancing (Nginx && LVS)
Load Balancing Algorithm
- polling
- WRR
- Load lowest priority
- Hash class (Hash operation performed according to key information, the Hash value is the same distribution request to the same server Hill)
Chapter VI distributed system design theory CAP
For distributed computing systems, can only meet two consistency, availability, fault tolerance three of partition.
Three two
consistency
All nodes can see the same data at the same time. (Such as MySQL Cluster consistency from master data)
Availability
Non-faulty node returns a reasonable response in a reasonable time. (Not a response to an error or timeout)
Partition fault tolerance
When the (network failure, network interruption loss) network partition occurs, the system can continue to run.
CAP details
CAP is the granularity of data, instead of the system
Such as user account data selection CP, while other information selected AP
Under normal circumstances, can simultaneously satisfy CA
When the partition happens, that is, network failure, there is only the choice of CA, under normal circumstances network, CA can meet simultaneously.
ACID：
Theory of database transaction integrity
- Atomicity: Either a single transaction have been completed, either failed (such as bank transfer, a subtraction, an addition, must be synchronized)
- Consistency: concurrent requests the data consistency
- Isolation: prevent multiple transactions concurrent data inconsistencies resulting from the implementation of the cross. (Optimistic and pessimistic locking lock realization)
- Durability: After the end of the transaction, changes to data is permanent, even if the system failure will not be lost.
BASE:
If you can not achieve strong consistency, then eventual consistency
Basically Availible basic available
When the distributed system failures, ensure that the core functions are available (stay logged available, the loss of registration)
Soft Status soft state
Data inconsistencies
Eventually Consistency eventual consistency
BASE theory is the extension application AP program.
Chapter VII of the store availability
Copy standby
- Storing host data, copying the data to a backup machine
- Preparation machine do not provide service to read and write
- Host failure require human intervention was promoted to the backup machine host
Master-slave replication
- Storing host data, copying the data to a backup machine
- Equipment machine provides reading services
- Host service fault can be read, played a standby machine performance
- From the master copy standby replication complex than, mainly in the master client needs to be identified from the relationship
- Applicable less reading, more writing systems (forums, news sites)
Standby switchover from the main switching and
Standby replication and host failures are the main issues from the common copy can not be written.
The switchover from switching to increase the role and the main switching functions based on the original.
Interconnects: transmitting channel status established between standby machine.
Channel can be connected to a network, it may be a serial connection.
Mediation type
Standby machine is not connected directly, but pass information via an intermediary. (The mediating high availability)
Zookeeper arbitration node settings node level.
Analog
The analog preparation machine as a client, the analog read and write operations.
Master-master replication
- Two host has data, synchronized by copying channel
- Great consistency
- For temporary and can be lost, you can cover the scene
Data dispersed clusters
Data dispersed cluster refers to a cluster multiple servers, each server will be stored as part of the data, at the same time, each server will be part of the data backup. (Sub-library sub-table)
Distributed transaction algorithm: consistency
2PC two-phase commit protocol
- The first stage: the coordinator sends a request (voting phase) to all participants (either negative participants can terminate submitted)
- The second stage: all participants by request, coordinator submit the request.
problem:
- Synchronous blocking: coordinator and participants wait for each other
- Coordinator single point of failure
3PC three-stage submission
- The first stage: the coordinator sends a request (voting phase) to all participants, participants have denied the transaction is aborted, all received yes, the second phase within the timeout period.
- The second stage: the coordinator of withholding sent to the participants, the participants received information on the enforcement branch operations, returns ACK message.
- After sending commit coordinator receives all ACK messages: the third stage. Participants perform after submitting returns have been submitted message to the coordinator.
A Distributed Consensus Algorithm
Paxos (purely theoretical)
Particularly complex
- Most of consistency
- Read algorithm will also be fully implemented again
Raft
- Leader election
- Log Replication
- ASD
BAR
Zookeeper distributed consensus algorithm used
Chapter VIII highly available computing
When the damaged part of the hardware, computing tasks can operate normally.
The basic idea: by adding more servers to achieve high availability computing.
Standby
- Host performs all computing tasks
When the host is damaged and can not be recovered, the need to manually raised to the standby host machine, and adding new preparation machine
Cold standby: packages and configuration files are ready, start the server, but the business does not start; warm standby: the business started, but does not provide services
Small number of users and other applicable internal management system, the background of the case management system
Master-slave
- Host operative tasks, task preparation machine operative
- Host failure, task distribution unchanged, even if the host is not working
- Promoted the need to manually prepare the host machine, and add a new backup machine
Symmetrical cluster (load balancing cluster)
- Under normal circumstances, the task allocator assign tasks to different hosts
- When a server fails, the task allocator will skip the server
- When the failed server is restored, reassign tasks
Asymmetric cluster
Master-Slave
- Cluster Server Role distinguished in some way, elected Master server
When the Master server fails, elected a new Master Server
Master protocol selected by ZAB Zookeeper
Chapter IX service availability
Places to live
Off room, engine room fire, City earthquake ... (but egg is not so easily broken) (Do not put all your eggs in one basket)
- Iso-city area
- Cross City places
- Transnational places
Design tips places to live
- Ensure that the core business of off-site live
- Eventually consistent core data (impossible to live off-site soon)
- Using various means to synchronize data
- message queue
- Second reading
- Reading and so on back to the source
- Ensure the most remote areas live (not 100%)
Remote live design steps
- Business Classification (selected core business)
- Data classification (the amount of data, the only, real-time, recoverability)
- Data Synchronization (storage system synchronization, the synchronization message queue, etc.)
- Exception handling (multichannel synchronization, sync, and access binding, logging, user compensation)
Interface-level fault response options
Relatively small room with a probability of fire, power failure, failure of the interface more often.
Interface-level fault:
- Internal: procedures, computer performance reached the limit, resulting in a database query slow
- External: hacker attacks, leading to panic buying promotional sudden increase in user traffic, slow response and other third parties
Demote
Downgrade is focusing on the high availability of the entire system, an act Diuju Paul handsome.
For example, the forum system close to the load, pause postings function, can only see posts.
(If the system continues to load, server crashes, see post features are also waste)
Fuse:
It is a situation downgrade. In other words, the fuse can cause degradation.
Fusing is a request reaches a threshold value, suspend the call to service, to prevent system overload, cause a crash.
Limiting
Only let some access. To ensure that all part of the response can not be better than the response.
- Based on the request flow restrictor (control threshold)
- Based on the resource limiting (current limit of critical resources, such as thread pool maximum concurrency)