Regarding the addition of orderer in raft mode, the official website gives an approximate process

reconfigure

Raft orderers support adding and removing nodes dynamically (meaning, while the channel is in use), but only one node at a time. Before you attempt to reconfigure, please note that your cluster must be in a state that can sustain maintenance and be able to achieve consensus. As a worst example, if you have 3 nodes and 2 nodes go down, you can't configure your raft cluster to remove nodes. Likewise, if you have 1 down node in a channel with 3 nodes, you should not try to replace the certificate, as this will cause a secondary error. * As a guideline , unless all consensusers are online and healthy, you should never attempt to make configuration changes to a Raft consensuser, such as adding or removing a consensuser, or replacing a consensuser's certificate, etc.

If you decide to modify these parameters, we recommend only trying it during a maintenance cycle. The vast majority of problems with modifying configurations occur in clusters with only a few nodes and one node goes down. For example, if you have three consensus nodes and one of them goes down, that means you only have two nodes alive. If you scale the cluster to 4 nodes in this state, you still only have 2 nodes alive, which doesn't allow for a quorum. The fourth node cannot come online, because nodes can only join a running cluster (cases where the total cluster size is 1 or 2).

So when scaling a three-node cluster (with only two nodes alive) to four nodes, you're completely stuck until the original offline node comes back up.

Adding a new node to a Raft cluster requires the following steps:

A channel configuration update transaction adds the new node's TLS certificate to the channel . Note: A new node must first join a system channel before joining one or more application channels .
Get the latest system channel configuration block from an orderer , which is part of the system channel.
Ensures that this node is part of the system channel by verifying that the configuration block contains the (to-be) joined node certificate .
Starts a new Raft node with the config block path specified in the General.BootstrapFileconfig parameter .
Wait for the Raft node to copy the block in the channel its certificate is joined to from an existing node . After this step is complete, the node starts serving the channel.
Add the newly added Raft node endpoint to the configuration of all channels.

Simply put, nodes that are already running (and have joined some channels) can be added to the channel at runtime. To do this, simply add the node's certificate to the channel's channel configuration. The node will automatically detect that it joins the new channel (the default value is 5 minutes, but if you want the node to detect the new channel faster, you can restart the node), then pull the channel block from the orderer in the channel, and finally create a new channel for the chain Start the Raft instance.

After successfully completing the above steps, the channel configuration can be updated to include the new Raft orderer's endpoint.

#查看这个介绍，我们可以感觉到官方在介绍这一部分时只给了大概流程，仅看这个的话很难操作。这是因为这类操作，官方在【动态新增组织】中进行了比较详见的流程介绍，主要区别在于
#1.在将orderer节点添加到指定应用通道前，需要先将orderer节点添加到系统通道中。
#2.修改配置区由peer修改为orderer。这部分需要自己寻找

Therefore, referring to the official introduction, the dynamic adding process can be designed as follows

Dynamically add process

Environment configuration

The device already has 3orderer nodes and 1 peer node

Generate orderer certificate

Fabric provides two ways to generate certificates fabric-ca generation and cryptogen tool generation, we use the cryptogen tool to generate

Modify the configuration file

The certificate generation file crypto-config.yaml adds the configuration of orderer4

OrdererOrgs:
  # ---------------------------------------------------------------------------
  # Orderer
  # ---------------------------------------------------------------------------
  - Name: Orderer
    Domain: example.com.local
    # ---------------------------------------------------------------------------
    # "Specs" - See PeerOrgs below for complete description
    # ---------------------------------------------------------------------------
    Specs:
      - Hostname: orderer1
      - Hostname: orderer2
      - Hostname: orderer3
      - Hostname: orderer4     #新增

generate certificate

fabric native command

cryptogen extend --input crypto-config  --config ./crypto-config.yaml

after allin transformation

#allin改造中使用证书合并模式。复制orderer1的证书，修改路径及证书名称

insert image description here

Prepare the channel configuration file containing the data to be updated

Option One

Modify the configtx.yaml file and add information about orderer4. This process mainly refers to the process of dynamically adding organizations. Automatically generate json files through fabric's own commands, which is highly automated

    SampleMultiNodeEtcdRaft:
        <<: *ChannelDefaults
        Capabilities:
            <<: *ChannelCapabilities
        Orderer:
            <<: *OrdererDefaults
            OrdererType: etcdraft
            EtcdRaft:
                Consenters:
                - Host: orderer1.example.com.local
                  Port: 7050 
                  ClientTLSCert: crypto-config/ordererOrganizations/example.com.local/orderers/orderer1.example.com.local/tls/server.crt
                  ServerTLSCert: crypto-config/ordererOrganizations/example.com.local/orderers/orderer1.example.com.local/tls/server.crt
                - Host: orderer2.example.com.local
                  Port: 8050 
                  ClientTLSCert: crypto-config/ordererOrganizations/example.com.local/orderers/orderer2.example.com.local/tls/server.crt
                  ServerTLSCert: crypto-config/ordererOrganizations/example.com.local/orderers/orderer2.example.com.local/tls/server.crt
                - Host: orderer3.example.com.local
                  Port: 9050 
                  ClientTLSCert: crypto-config/ordererOrganizations/example.com.local/orderers/orderer3.example.com.local/tls/server.crt
                  ServerTLSCert: crypto-config/ordererOrganizations/example.com.local/orderers/orderer3.example.com.local/tls/server.crt
                - Host: orderer4.example.com.local                                                                                        #新增
                  Port: 10050                                                                                                             #新增
                  ClientTLSCert: crypto-config/ordererOrganizations/example.com.local/orderers/orderer4.example.com.local/tls/server.crt  #新增
                  ServerTLSCert: crypto-config/ordererOrganizations/example.com.local/orderers/orderer4.example.com.local/tls/server.crt  #新增
            Addresses:
                - orderer1.example.com.local:7050 
                - orderer2.example.com.local:8050 
                - orderer3.example.com.local:9050
                - orderer4.example.com.local:10050                                                                                         #新增
            Organizations:
            - *OrdererOrg
            Capabilities:
                <<: *OrdererCapabilities
        Application:
            <<: *ApplicationDefaults
            Organizations:
            - <<: *Org1
        Consortiums:
            SampleConsortium:
                Organizations:
                - *Org1

After modifying the test, it is found that the commands provided by fabric only support organization-level selection, but the part we need to modify is that the Profiles module cannot be used.

    configtxgen -configPath $FABRIC_CFG_PATH/configs -printOrg ${SET_MSP_ID} >$FABRIC_CFG_PATH/configs/${SET_PEER_ORG_NAME}.json
     -printOrg    string   将组织的定义打印为JSON。(对于手动向通道添加组织非常有用)

Option II

Do not prepare the data to be added in advance, when updating, directly generate and then update to the corresponding file

Get the current channel profile

set environment variables

export FABRIC_CFG_PATH=/root/raft/configs
export CORE_PEER_LOCALMSPID=Org1MSP
export CORE_PEER_MSPCONFIGPATH=${FABRIC_CFG_PATH}/crypto-config/peerOrganizations/org1.example.local/users/[email protected]/msp
export CORE_PEER_ADDRESS=peer1.org1.example.local:7051
export CORE_PEER_TLS_ENABLED=true
export CORE_PEER_TLS_ROOTCERT_FILE=${FABRIC_CFG_PATH}/crypto-config/peerOrganizations/org1.example.local/peers/peer1.org1.example.local/tls/ca.crt
export CORE_PEER_TLS_KEY_FILE=${FABRIC_CFG_PATH}/crypto-config/peerOrganizations/org1.example.local/peers/peer1.org1.example.local/tls/server.key
export CORE_PEER_TLS_CERT_FILE=${FABRIC_CFG_PATH}/crypto-config/peerOrganizations/org1.example.local/peers/peer1.org1.example.local/tls/server.crt
export ordererCa=${FABRIC_CFG_PATH}/crypto-config/ordererOrganizations/example.local/orderers/orderer1.example.local/msp/tlscacerts/tlsca.example.local-cert.pem

注意：新节点在加入一个或更多应用通道前，必须先加入到#系统通道 即syschannel
所以在文档中以syschannel为例，讲解流程。在加入应用通道是，需要经syschannel 更换为对应应用通道的名称

Pull the latest configuration block file

config_block.pb

peer channel fetch config $FABRIC_CFG_PATH/config_block.pb -o orderer1.example.com:7050 -c syschannel --tls --cafile $ordererCa

Extract and convert formats

Extract valid data from config_block.pb and convert it into editable json format

configtxlator proto_decode --input $FABRIC_CFG_PATH/config_block.pb --type common.Block | jq .data.data[0].payload.data.config > $FABRIC_CFG_PATH/config.json

Modify the orderer configuration

In the generated json file, write the information of orderer4 (the certificate-related path should be changed to the format of the certificate file itself (base64)) in the corresponding newly added position in Solution 1. Get the new json file config_updated.json

//此处应有图，脚本编写完成,实际执行成功后补

build new configuration

#把前后的两个json文件重新转换回pb类型文件 
configtxlator proto_encode --input $FABRIC_CFG_PATH/config.json --type common.Config > $FABRIC_CFG_PATH/config_block_origin.pb
configtxlator proto_encode --input $FABRIC_CFG_PATH/config_updated.json --type common.Config > $FABRIC_CFG_PATH/config_block_updated.pb
#比较前后两个pb文件的差异，得到改动部分 
configtxlator compute_update --channel_id byfn-sys-channel --original $FABRIC_CFG_PATH/config_block_origin.pb --updated $FABRIC_CFG_PATH/config_block_updated.pb > $FABRIC_CFG_PATH/config_diff.pb
#把配置变化部分转化为json文件
configtxlator proto_decode --input $FABRIC_CFG_PATH/config_diff.pb --type common.ConfigUpdate > $FABRIC_CFG_PATH/config_diff.json
#为上述json文件添加头部信息(Header)，封装成一个完整的config update请求
echo '{"payload":{"header":{"channel_header":{"channel_id":"byfn-sys-channel", "type":2}},"data":{"config_update":'$(cat $FABRIC_CFG_PATH/config_diff.json)'}}}' | jq . > $FABRIC_CFG_PATH/config_diff_envelope.json
#把封装好的json文件转换回pb格式文件
configtxlator proto_encode --input $FABRIC_CFG_PATH/config_diff_envelope.json --type common.Envelope > $FABRIC_CFG_PATH/config_diff_envelope.pb

sign

peer channel signconfigtx -f $FABRIC_CFG_PATH/config_diff_envelope.pb

Submit an update request

peer channel update -f $FABRIC_CFG_PATH/config_diff_envelope.pb -c byfn-sys-channel -o orderer.example.com:7050 --tls true --cafile $ORDERER_CA

Finish

After the system channel has been added, the channel name should be changed to the name of the application channel (you want to join), and the appeal procedure should be performed again.

Start the orderer4 node

The process of starting the orderer node is not much different from the original startup method, but there are a few points that need attention and resolution

1.新节点的hosts更新
2.peer的客户端维护的orderer列表里，需要把新增的orderer更新进去

Check the log, if it starts normally and starts synchronizing blocks (of all joined application channels), it means that the whole process is completed normally

reference documents

Official document reconfiguring RAFT: https://hyperledger-fabric.readthedocs.io/en/latest/raft_configuration.html#reconfiguration

Dynamic new organizations in official documents: https://hyperledger-fabric.readthedocs.io/en/latest/channel_update_tutorial.html

Fabric 1.4.2 dynamically adds orderer nodes: https://www.jianshu.com/p/49a915bed152

Fabric dynamically configures Raft nodes: https://www.cnblogs.com/cbkj-xd/p/12123860.html

Dynamically add orderer process design

In the boundary of the product, we have settings that need to be followed. There are a few points that need to be followed in this design

1、每台机器上的orderer节点不允许超过3个。
2、sdk（即peer的客户端），需要动态的更新最新的orderer列表，以便可发送消息给最新的orderer节点
3、私钥不出设备

Under the premise of complying with the above restrictions, carry out process design.

execution environment

During the pre-research process, we found that the main factors that determine the process execution environment, certificate generation and signature authority all point to the node where the Admin of the orderer organization is located.

Signature authority

Although we can grant Admin permissions to all orderer members by modifying the permissions policy. But we cannot share the orderer's private key on the chain to a device without an orderer node.

All, our execution object should be

1.如果不将Admin权限下放，执行对象应该是orderer组织Admin在的设备，即联盟的第一台设备。
2.如果将权限下放给组织成员，那执行对象也至少要是一个已存在orderer节点的设备，而不是任意一台机器。

This requires us to maintain a list of available orderers on the chain, and set a rule for when to choose which orderer's device should be used as the execution environment.

certificate generation

Because the private key does not come out of the device, there is no orderer node on the new device. It becomes impossible to directly share the existing orderer certificate.

In this case, you can only use CSR to apply for a certificate from the orderer organization ca to prepare the orderer private key and certificate in advance on the new device.

In this case, the step of generating a certificate must be reserved in the process, and the difference between the two processes cannot be explained to the user (the orderer user cannot perceive it).

And the current product is mainly adapted to the csr-ca-cert certificate scheme, so even if the orderer is added on the device with the existing orderer, the certificate sharing mode is no longer selected.

vote

Refer to the design of dynamically adding nodes in the configuration center when members in the organization join. It only needs to be agreed once by the administrator node in the organization.

In this way, in the case of non-dynamically adding a new organization orderer, the selected execution device can directly perform the signature operation without voting. Consider that the orderer should try not to open the concept to users. In this case, the voting process will not be added for the time being.

sdk update

In order for the newly joined node to obtain the IP of the device where the orderer is located, in order to set up its own hosts. After the first device initializes the network, the orderer list is uploaded to the chain. The uplink data is the device ip and the ca/tlsca certificate of orderer1 (for channel creation). So when the new orderer completes the operation, we can know the current devices that need to be updated in a very simple way, send a notification to the ip in the list, and the corresponding device will go to the chain to obtain the latest orderer list after receiving the notification. Update the list of orderer users in your own hosts and sdk.

It should be noted that updating the sdk user list should also be dynamic, and it should be done without affecting normal use

Fabric dynamically adds orderer