Database Middleware Architecture Series real - architecture implementation

Overall structure

• Proxy CLIENT and connected to the main MYSQL, communication is critical to a communication component Netty, MYSQL protocol package, the main problem to solve stick package and unpacking
• MYSQL protocol layer, protocol parsing mysql, primarily responsible for the received TCP packet Netty according MYSQL protocols parsed into the SQL statement; there is the combined processed SQL statement parsed into MYSQL sent to the specified protocol where
• After MYSQL protocol parsed into SQL statements, using MYSQL SqlParser components will resolve into a syntax tree, get information of various parts of SQL
• SqlRouter then read sub-library components or other policy configuration sub-table is rewritten SQL statement, and routing node
• SqlExecutor then these components into SQL, select the appropriate execution strategy, e.g. thread pool, fences, semaphores
• Finally, the need to perform these SQL is transmitted to the communication layer through MYSQL
• After mysql after processing is completed, the returned data is received through the communication layer, SQLExecutor component execution, the final result may need to be incorporated by ResultMerger assembly, and then sent back to the client through a communication layer

MYSQL agreement

mysql protocol packet basis

• Loaded data packet payload length is the number representative of this byte, 24 = 16M maximum power 2
• Sequence_id packet sequence, once out of order, mysql client will report an error, this particular agreement in ResultMerger the process of restructuring that need attention, a lot of this when I planted a somersault in it
• actual payload data transmission

Connection Agreement

Mysql client and itself a handshake agreement only if Mysql agreement after sending OK, Client can send SQL statements

• When I want to send a request to limit the amount of MYSQL, then you need to parse SQL statements, you only need to add some restrictions during the forwarded;
• Signal can be used to control the amount of traffic to MYSQL, can dynamically control the load MYSQL
• SQL can be masked some dangerous

Note: The number of connections is not reduced

• In order to reduce the number of connections, you must achieve a connection pool, pay attention to keep alive policies connected connection pool, MYSQL default every eight hours will automatically kill off connection
• Each connection pool before connecting to join must first handshake agreement
• For the Client, to be modeled as the PROXY MYSQL communication, communication for MYSQL, to simulate the PROXY to Client
• PROXY saves all the data sources MYSQL password, to the same encryption method stored MYSQL
• When executing a SQL statement, select a connection from the connection pool to perform, after the execution is over, immediately returned back to the connection pool. This can greatly reduce the number of connections MYSQL.
• If you execute SQL statements in a transaction which, then the connection can not be released after the completion of the transaction must be returned back to the connection pool

Verification process

• 公式：Client_Secure_Password= SHA1( password ) XOR SHA1( “20-bytes random data from server” SHA1( SHA1( password ) ) )
• MYSQL to store passwords: Server_Password = SHA1 (SHA1 (password))
• Corollary: Stash_Password = SHA1 (password)
• Verify: SHA1 (Stash_Password) = = Server_Password?

1. "20-bytes random data from server", the data generated PROXY is sent to the client
2. Server_Password, this is based password generation, and the configuration of the PROXY
3. Client_Secure_Password, this data, mysql client generates a random number according to 20bit, then sent to the PROXY

MYSQL Query Protocol

• column definitions include field-count, multiple field, there will be a final EOF, indicating the end of
• row definitions includes a plurality of row data, and finally there will be a EOF, indicating the end of

Netty threading model

Threading model

Early threading model

• Scheduling and execution mixing level is not clear
• Concurrent processing is too complex
• Error handling is too cumbersome

Now threading model

• Scheduling and performing separation, only a pool of execution
• All the actions performed task session Encapsulate
• Serial task execution the same session

Worker thread pool implementation

• Runnable task inheritance and comparable interfaces
• SchedulerWorker there are n connection pool, session creating a corresponding index calculated
• All business logic interfaces are inherited manager
• Using a priority queue, implemented task priority