Article Directory
Foreword
Previous article ( Pipeline status Ozone SCM transformation analysis ), the author of the state of the SCM Pipeline converted done some analysis. Detailed analysis of the author of this article will be another important concept in SCM Container done. Container storage unit serving as Ozone SCM, Replication internal SCM Container level in the data, the heartbeat reporting operation and the like. This is similar to the process for HDFS NameNode within the Block. In this paper the author describes the state of the Container transformed, as the form of data within the organization's Container, in front of the author describes the SCM Datanode article has been introduced, will not expand the space here set forth.
Container processing between the SCM and Datanode
Because Ozone SCM provides data storage units Container, therefore Datanode node service center under the management of SCM and its report is based on heartbeat Container Report.
Which process involves as follows:
1) Datanode get local Container latest status report, send heartbeat way to SCM services.
2) SCM Datanode service after receiving the report of Container, Container compare information maintained by its own memory, to update the relevant Container state while addressing inadequate number of copies and miss out Container copies.
3) SCM internal ReplicationManager Container service will also regularly check the copy status information, a copy of Container processing, such as copies of redundant, inconsistent and inadequate conditions of Container state, it is necessary to trigger the operation close Container commands.
4) SCM of Stale / DeadHandler dead processing nodes, when inactive Pipeline, will trigger a Container close command in dead node Container close operations within inactive Pipeline contained.
Illustrating a process of the above process are as follows:
SCM Container state conversion
Container in the process between the SCM and the Datanode, will involve multiple switching state Container.
Here, however, we have a switching Container Pipeline more complex than the state of the process, which contains a total of the following six states, transformed with the corresponding event transient event definitions different.
- OPEN, Container initially create the initial state, Container in the OPEN state can only write data to conduct its internal operations.
- CLOSING, Container is in a state of being closed.
- QUASI_CLOSED, Container pending off state.
- CLOSED, Container in a closed state, the only state in the CLOSED Container in order to be allowed to perform replication operations. An external operation can read the internal data Container CLOSED state.
- DELETING, Container Deleting state.
- DELETED, Container deleted state, Container deletion is present mainly in the admin user to delete unwanted copies Container operational behavior and deletion processing operations.
Under the above state where the focus of distinction QUASI_CLOSED and CLOSED. Container QUASI_CLOSED before the CLOSED state in a temporary transient, the intention is Container coming off state. Because in actual Container Close operation, there will Raft coherence protocol communications due to network or node failures lead to consistency can not be reached, but at this time we do not make the Container has been in a state of OPEN available, so we need to have a means of you can be allowed to become CLOSED state. Container becomes newly written after the CLOSED state, replication operation can be performed, and the data will not be made Container operated.
Given this situation, the introduction of community QUASI_CLOSED this state, the first state from the Container CLOSING state to QUASI_CLOSED. If the report is up more than half of the copies QUASI_CLOSED, then SCM side can force the Container again becomes CLOSED. Container Datanode update on the status of the Datanode the next heartbeat, will be reported to the SCM side.
FIG handover procedure between the respective states Container as follows:
Quote
[1].https://issues.apache.org/jira/browse/HDDS-2794 . Failed to close QUASI_CLOSED container
[2].https://issues.apache.org/jira/browse/HDDS-1561 . Mark OPEN containers as QUASI_CLOSED as part of Ratis groupRemove
[3].https://issues.apache.org/jira/browse/HDDS-737 . Introduce Incremental Container Report
