Article Directory
Foreword
In the Ozone, Container state consistency as to update the node Pipeline Organization StateMachine of the form. But here the face of some of the edge cases that may be present, such as sudden restart Pipeline node, accidental deletion Container directory, we need to have other means to show the latest status Container is currently located. Thus we can know whether a state Container stored in the copy on the Pipeline between nodes is consistent, consistent Block contains the data, and so on. To this end, the community introduced a self-energizing type transactionId in Container, indicating the current status of the latest Container. The author of this article to talk about the self-energizing type transaction ID in the implementation and use of Ozone Container.
Ozone Container自增型TransactionId：BlockCommitSequenceId
Here I want to first introduce introduce Ozone Container Id increment the origin of the type. In the Ozone, Block Container is provided for external data storage unit, which will accept the data from the internal read request to the external Block. In brief, a Container in the case of its space to spare, there will be a steady stream of Block external data writing. Given the delay machine where there may be different copies of the same Container communications, networking, and data write speed there will be problems, and therefore Container at the same time in the latest "data status" will be different. If we each operation as a Transaction, then the implementation of the latest Transaction Container will be the latest "data state." Ultimately, however, a copy of which will be executing the Container Transaction write operation, and then reach a final agreement "data state."
So the question is, in Ozone, how can we achieve this TransactionId it? Id operator needs to ensure that each node in the Container Pipeline having unique properties. In Ozone in, Datanode is to achieve consistency Container operation is controlled by Apache Ratis + ContainerStateMachine way, and ContainerStateMachine itself will apply from Raft log Raft server, we can take full advantage of Raft log id as the Container Transaction update Id. In this case, we do not have another generation of exclusive TransactionId Container. Here, Ozone this TransactionId called BlockCommitSequenceId, meaning Id after each Block data written submission.
BlockCommitSequenceId follow the following principles:
The final BlockCommitSequenceId multiple copies of the same Container is consistent, meaning the final Container "data state" consistency. If there are less than the current date BlockCommitSequenceId Container id transaction at the time of writing, it explained that it had been written before Block this data, you can ignore this write operation.
BCSID (BlockCommitSequenceId short) in addition to updating the Container level, it will also be recorded in each Block of data, as an attribute of Id stored in Block information.
This conversion process is shown in FIG follows:
The figure shows BCSID addition maintains the latest value of a Container on the outside, Block data is written in each also having a respective BCSID.
BlockCommitSequenceId intended use
BCSID is recorded and updated in a Container, then what it has additional intended use of it?
The usage scenario in BCSID Ozone, which has at least the following purposes:
- Determine whether the data state between Container replica is consistent. For example, in doing Container replication operation when checking existing Container copies of state BCSID and SCM management service center maintained by BCSID whether, and if not, the operation is not performed Container Replication. If yes, the operation performed Container is closed, and the subsequent operation Replication.
- Container to determine whether Datanode local state data changes. If the data changes occur Container status Datanode before and after the restart, it BCSID will change, if the value of the current record of BCSID before the restart, after restart, it loads BCSID find different values, then the data has changed. This data indicates that this change has occurred Container, unhealthy state.
Quote
[1].https://issues.apache.org/jira/browse/HDDS-1843
[2].https://issues.apache.org/jira/browse/HDDS-935
[3].https://issues.apache.org/jira/browse/HDDS-603
