checkpoint mechanism in both environments Hadoop

Pseudo-distributed environment:

HA environment checkpoint mechanism

An HA of HDFS, there are two active and STANDBY namenode namenode nodes. Their memory is saved in the same cluster metadata information, as standby namenode already in memory, so state storage to create a cluster checkpoint checkpoint process to generate a new fsimage just from memory.

As detailed below: (standby namenode = SbNN, activenamenode = ANN)

1. SBNN see if the conditions to create a checkpoint to meet:

(1) Time since last checkpoint interval> = $ {dfs.namenode.checkpoint.period}
number of transactions strip (2) Edits reaches $ {dfs.namenode.checkpoint.txns} limits
of these conditions is either met, it triggers the creation of a checkpoint.

2. SbNN will save the current state into memory in a new file named fsimage.ckpt_txid. Wherein txid is ID (transaction ID) of the last edit of the last transaction. Then create a file for fsimage MD5 file and rename the file to fsimage fsimage_txid.

3. SbNN active namenode sends an HTTP GET request. Txid request includes a domain name, port, and the new fsimage of SbNN.

4. ANN receipt of the request, with the information obtained in turn re-transmits an HTTP GET request to SbNN, fsimage obtain a new file. After the new fsimage file transfer to the ANN, it is also the first named fsimage.ckpt_txid, and create a MD5 file for it. And then changed its name to fsimage_txid. fsimage process is complete.

 



Guess you like

Origin www.cnblogs.com/TiePiHeTao/p/9959615e239ec4e5652468d4a00b82e5.html