Postgres XL consists of 3 main components: GTM, Coordinator and Datanode.
GTM is responsible for providing the ACID properties of transactions.
Datanodes store table data locally and process SQL statements.
The Coordinator processes each SQL statement from Applications, decides which datanode to execute, and sends the execution plan to the corresponding datanode.
Usually GTM needs to run on a separate server, because GTM needs to handle transaction requests from all coordinators and datanodes. In order to reduce data interaction, the requests and responses from the coordinator and datanode of the same server are grouped into the same group, which can be achieved by configuring GTM-proxy. GTM-proxy is actually a proxy, and coordinator and datanode are originally connected to GTM directly Yes, you can group multiple coordinators, each with a GTM-proxy, the coordinator is connected to the GTM-proxy, and the GTM-proxy reduces the number of interactions and data volume to GTM by sending requests in batches (similar to aggregation requests, reducing the number of requests). . GTM-proxy also helps in handling GTM node failures.
Once GTM fails, the entire cluster is immediately inaccessible, and you can switch to the GTM Standby node at this time. If the GTM Standby node is deployed, the GTM Proxy should be deployed at the same time, which is generally deployed on the same server as the Coordinator and Datanode. The role of GTM Proxy is to act as a proxy for the access of Coordinator and Datanode to GTM, and to reduce the load of GTM. Another important role is to help complete the failover of GTM. When the GTM node fails, GTM Standby becomes the new GTM. The Coordinator and Datanode nodes do not need to re-assign the GTM address, only the GTM Proxy needs to reconnect to the new GTM address.
A better way is to run the datanode and coordinator components on the same server, so that you don't have to worry about the load balancing between the two. You can get data from the local replication table without sending requests to the network. The datanode and coordinator component groups can be run on multiple servers. The datanode and coordinator components are essentially PostgresSQL instances, and different working directories and ports need to be configured to avoid conflicts.
The coordinator does not store user data, it only stores catalog data, which is used to decide how to process statements, go to that datanode for execution, and generate local SQL statements for each datanode. Therefore, there is no need to worry about the failure of the coordinator component, just switch to another one. The node where the coordinator receives data access requests is essentially composed of PG background processes. After receiving a query, the Coordinator node executes the query plan, and then distributes the query to the relevant data nodes according to the data nodes involved in the query data. When writing data, it will also write data to related nodes according to different data distribution strategies. It can be said that the Coordinator node holds the global data location of the cluster. The Coordinator node can be expanded arbitrarily, and each node is completely equal except for the different access addresses. The data updated by one node can be seen immediately on another node. Each Coordinator node can be configured with a corresponding standby node to avoid a single point of failure.
Datanode is the node that actually accesses data. It receives requests from Coordinator and executes SQL statements to access data. Nodes also communicate with each other. Generally, the data on a node is not global, and the data node does not directly provide data access to the outside world. There are two modes for the distribution of data of a table on data nodes: replication mode and sharding mode. In replication mode, the data of a table has multiple copies on the specified node; Certain rules are distributed on multiple data nodes, and these nodes jointly save a complete data. The choice of these two modes is specified by executing the CREATE TABLE statement when the table is created. The specific syntax is as follows:
CREATE TABLE table_name(...)
DISTRIBUTE BY
HASH(col)|MODULO(col)|ROUNDROBIN|REPLICATION
TO NODE(nodename1,nodename2...)It can be seen that if DISTRIBUTE BY is followed by REPLICATION, it is replication mode, and the rest is fragmentation mode. HASH refers to the distribution of data according to the hash value of the specified column, and MODULO refers to the distribution of data according to the friction operation of the specified column. , ROUNDROBIN refers to the distribution of data in a polling manner. TO NODE specifies the node range of data distribution. If not specified, all data nodes participate in data distribution by default. If no distribution mode is specified, that is, using a normal CREATE TABLE statement, PGXL will use the sharding mode to distribute data to all data nodes by default.
GTM is a single point of failure. The state of GTM can be backed up by configuring GTM-slave. When GTM fails, GTM-proxy can switch to the slave node.
The entire cluster consists of GTM, GTM-Proxy, Coordinator, and Datanode.
GTM (Gloable Transaction Manager) is responsible for providing ACID properties of transactions;
Datanode is responsible for storing table data and locally executing SQL tasks dispatched by Coordinator;
The Coordinator is responsible for processing each SQL task from the Application, and decides which Datanode will execute it, then dispatches the task plan to the corresponding Datanode, and collects the results and returns them to the Application as needed;
GTM is usually undertaken by an independent server, and GTM needs to process transaction requests from all GTM-Proxy or Coordinator and Datanode.
It is best to configure a Coordinator, a Datanode and a GTM-Proxy on each machine at the same time.
Each machine is configured with a Coordinator and a Datanode at the same time, which can load balance and reduce network traffic at the same time. GTM-Proxy will reduce the GTM load, aggregate requests and responses from processes on the Coordinator and Datanode onto a single machine, and help deal with GTM failures.
GTM may have a single point of failure, and a GTM-Standby node can be configured as a standby node for GTM.
2.1 Coordinator
Handle client connections.
Analyze the query statement, generate an execution plan, and pass the plan to the data node for actual execution.
Perform final processing on the query intermediate result set returned by the data node.
Managed transactional two-phase commit (2PC).
Store global catalog (GlobalCatalog) information.
2.2 Data Node (DataNode)
The table and index data are actually stored, and the data is automatically scattered and distributed (or copied) to each data node in the cluster.
Only the coordinator is connected to the data nodes to be readable and writable.
Execute the query downloaded by the coordinator, a query is queried in parallel on all relevant nodes.
A one-to-one communication connection can be established between two data nodes to exchange information related to the associated query of the distributed table.
2.3 Global Transaction Manager (GTM)
Global Transaction Manager (GTM: Global Transaction Manager)
There is only one GTM node in the entire cluster, and there will be a single point of failure. You can configure the StranBy hot standby node to ensure high availability.
Solve GTM performance bottlenecks by deploying GTM Proxy.
Provides a consistent view between transactions.
Handle the necessary MVCC tasks:
transaction IDs Transaction IDs.
snapshots Data snapshots, used by MVCC.
To manage global data values:
timestamps Timestamps.
sequences sequence object.
2.4GTM Proxy
Ø Works with the Coordinator and the DataNode.
Ø The coordinator and data nodes directly interact with GTM Proxy instead of GTM, which acts as the middleman between the backend and GTM.
Ø Group the requests to GTM, and submit multiple requests to GTM at one time.
Ø Get the range of transaction ids (XIDs).
Ø Obtain data snapshots.
2.5 Data distribution
There are two modes of data distribution: Replicated Table and Distributed Table
Replicated Table: Each row of records is replicated to all data nodes in the cluster, one copy per node.
Distributed Table: There are different nodes for record sharding, and the available sharding strategies are Hash, Round Robin, and Modulo.
2.6 High Availability
The global transaction manager adopts the hot standby mode.
Load balancing among multiple coordinators.
The data node uses streaming replication to replicate data to the standby node.
Reference:
https://www.postgres-xl.org/documentation/tutorial-arch.html
https://www.linuxidc.com/Linux/2016-11/137238.htm