How to scan blocks in Ethereum, Magician-Web3 1.0.5 is released, adding load balancing and retry strategies

Magician-Web3 is a blockchain development kit. It consists of two functions. One is to scan the blockchain and monitor transactions as needed by the developer. The other is some secondary packaging of web3j, which can reduce the workload of developers in some common scenarios. It plans to support three chains, ETH (BSC, POLYGAN, etc.), SOL and TRON

point of this update

1. Added load balancing, as long as multiple RPC URLs are configured, automatic polling can be performed, and the traffic can be distributed to multiple nodes, without having to keep staring at one node for "scourge"

2. The retry strategy has been added. When some unexpected reasons cause the block to be skipped for scanning, the height of the skipped block will enter the retry strategy, and you can handle it yourself

3. Modified a little detail

load balancing

Just set multiple rpc addresses directly

MagicianBlockchainScan.create()
                    .setRpcUrl(
                            // 可以设置1到n个 rpc地址了，超过一个将自动轮询使用
                            EthRpcInit.create()
                                    .addRpcUrl("https://data-seed-prebsc-1-s1.binance.org:8545")
                                    .addRpcUrl("https://data-seed-prebsc-2-s1.binance.org:8545")
                                    .addRpcUrl("https://data-seed-prebsc-1-s2.binance.org:8545")
                    )
                    .setScanPeriod(1000)
                    .setBeginBlockNumber(BlockEnums.LAST_BLOCK_NUMBER.getValue())
                    .addEthMonitorEvent(new EventOne())
                    .addEthMonitorEvent(new EventThree())
                    .setRetryStrategy(new EthRetry())
                    .start();

retry policy

When the following two conditions are met, the retry strategy will be triggered. Both conditions must be met to trigger a retry

1. The height of the block currently being scanned is empty (the block does not exist or there is no transaction in the block)

2. The block height currently being scanned < the latest block height on the chain

When the above two conditions are met at the same time, the scanning task will skip this block and continue to scan the next block. At the same time, the retry strategy will receive the height of the skipped block. You can handle it yourself in the retry strategy

Create a retry policy

public class EthRetry implements RetryStrategy {

    @Override
    public void retry(BigInteger blockNumber) {
        
    }
}

Add retry policy to scan task

MagicianBlockchainScan.create()
        .setRetryStrategy(new EthRetry())// 调用这个方法添加
        .start();

Need to pay attention to the configuration of the number of threads

If you open a block scan task + a retry strategy at this time, it will take up two threads, so the parameter must be passed 2

// 初始化线程池，核心线程数必须 >= 扫块的任务数量 + 重试策略的数量
EventThreadPool.init(2);

detail adjustment

1. The scanning frequency can be set as low as 500 milliseconds

2. The parameter passing type for setting the RPC address has changed

3. Due to the optimization of the second point, now you can judge what chain you want to scan according to the type of RPC address, so the setting of ChainType is removed

MagicianBlockchainScan.create()
                    .setRpcUrl(// 这里发生了改变
                            EthRpcInit.create()
                                    .addRpcUrl("https://data-seed-prebsc-1-s1.binance.org:8545")
                                    .addRpcUrl("https://data-seed-prebsc-2-s1.binance.org:8545")
                                    .addRpcUrl("https://data-seed-prebsc-1-s2.binance.org:8545")
                    )
                    .setScanPeriod(1000)// 这个参数最低可以设置为500了
                    .setBeginBlockNumber(BlockEnums.LAST_BLOCK_NUMBER.getValue())
                    .addEthMonitorEvent(new EventOne())
                    .addEthMonitorEvent(new EventThree())
                    .setRetryStrategy(new EthRetry())
                    .start();

You can visit the official website to learn more: https://magician-io.co