NEO looks at UTXO transactions from source code analysis

0x00 Preface

Community boss: "Transaction is the only way to operate the blockchain."

0x01 Transaction Type

In NEO, almost all operations on the blockchain except consensus are a "transaction", and even in front of the "transaction", the contract is just a little brother. The definition of transaction type is in TransactionType in Core:

Source code location: neo/Core/TransactionType

        /// <summary>
        /// 用于分配字节费的特殊交易
        /// </summary>
        [ReflectionCache(typeof(MinerTransaction))]
        MinerTransaction = 0x00,

        /// <summary>
        /// 用于分发资产的特殊交易
        /// </summary>
        [ReflectionCache(typeof(IssueTransaction))]
        IssueTransaction = 0x01,
        
        [ReflectionCache(typeof(ClaimTransaction))]
        ClaimTransaction = 0x02,

        /// <summary>
        /// 用于报名成为记账候选人的特殊交易
        /// </summary>
        [ReflectionCache(typeof(EnrollmentTransaction))]
        EnrollmentTransaction = 0x20,

        /// <summary>
        /// 用于资产登记的特殊交易
        /// </summary>
        [ReflectionCache(typeof(RegisterTransaction))]
        RegisterTransaction = 0x40,

        /// <summary>
        /// 合约交易，这是最常用的一种交易
        /// </summary>
        [ReflectionCache(typeof(ContractTransaction))]
        ContractTransaction = 0x80,

        /// <summary>
        /// 投票合约 //votingDialog
        /// </summary>
        [ReflectionCache(typeof(StateTransaction))]
        StateTransaction = 0x90,
        /// <summary>
        /// Publish scripts to the blockchain for being invoked later.
        /// </summary>
        [ReflectionCache(typeof(PublishTransaction))]
        PublishTransaction = 0xd0,
        /// <summary>
        /// 调用合约   GUI invocatransactiondialog
        /// </summary>
        [ReflectionCache(typeof(InvocationTransaction))]
        InvocationTransaction = 0xd1

These transactions not only have many names, but also the actual functions are somewhat different from what I "thought". In order to find out what each transaction does, I almost turned over the source code of NEO and GUI again.

• MinerTransaction : Miner fee transaction, this special transaction is added by _speaker_ when a new block is created. Use location: neo/Consensus/ConsensusService.cs/CreateMinerTransaction.
• IssueTransaction : Asset distribution transaction, a special transaction used to distribute new assets when they are created. Use location: neo/Core/BlockChain.cs/GenesisBlock.
• ClaimTransaction : NEOGAS fetch transaction, used to fetch GAS. There is an "Advanced" tab in the main interface of the GUI. After clicking, there is an extract GAS in the drop-down list, which is called this transaction. Use location: neo-gui/UI/ClaimForm.cs/button1_Click.
• EnrollmentTransaction : It is used to apply to become a bookkeeper, that is, a transaction for a member of parliament. This transaction GUI does not have an access interface, which means that there is no way to apply to become a bookkeeper using the GUI. Quietly said: If you want to be a bookkeeper, you must have at least several hundred million worth of NEO, so if you are a bookkeeper, you should not be stingy to transfer a few GAS to my NEO account.
• RegisterTransaction : The transaction of registering new assets, which was also mentioned in my previous blog "UTXO Assets from NEO Source Code Analysis". NEO and GAS were created in the genesis block through this transaction. But what is interesting is that this transaction is only used when registering NEO and GAS. The way to register new assets in the GUI is to make contract-level system calls, and create new assets by calling "Neo.Asset.Create".
• ContractTransaction : I think the most confusing thing is this contract transaction. I always thought that this should be used when publishing application contracts. After all, it is called a contract transaction, but goose, too young too simple, this transaction is called when we transfer money, yes, every transfer is a contract For transactions, you can see the transaction type in the transfer history of the GUI, which is basically this contract transaction. Use location: neo/Wallets/Wallet.cs/MakeTransaction.
• StateTransaction : As stated in the white paper, the members of the NEO network are elected through elections, which means that the general public can decide the structure of the superstructure and participate in the construction of socialist modernization. This is the power given to us by NEO. In the GUI, you can right-click the account and select Vote to participate in the election vote. Where to use: neo-gui/UI/VotingDialog.cs/GetTransaction.
• PublishTransaction : This transaction is the real _application contract_ publishing transaction, that is to say, in theory, if we want to publish the contract to the blockchain, we need to broadcast our contract through this PublishTransaction type of transaction. However? The reality is cruel, the GUI does not call this transaction, and publishing a new contract is still a contract-level system call, and a new contract is created by calling the "Neo.Contract.Create" API of the vm. The code location for deploying the application contract: neo-gui/UI/DeployContractDialog.cs/GetTransaction.
• InvocationTransaction : The type of evil transaction that replaces the role of our poor RegisterTransaction and PublishTransaction, contract invocation transaction. In the transaction after we successfully deployed the contract, InvocationTransaction is written in the type column of the GUI transaction history. The way to construct a transaction is even more simple and rude, that is to directly call EmitSysCall and then pass in the parameters to finish. Where to use: neo-gui/UI/AssetRegisterDialog.cs/GetTransaction.

The above are the 9 transaction types in NEO, which are basically all the operational means that can affect the generation of the blockchain except for the speaker to create a new block. Although I have roughly analyzed each transaction type, there are still some questions that are not clear, such as why a system call is made instead of sending the corresponding transaction when creating a new asset and deploying a contract. I did not find the answer from the source code.

0x02 balance

As can be seen from my title, I am not going to introduce each transaction type in detail. After all, in essence, they are just contracts with different scripts. What I want to analyze in detail here is the UTXO transaction, which is the contract transaction, and it is also our daily transfer operation on the NEO network. I often hear people in the community asking: "What exactly is a NEO account?" How did the balance of NEO and GAS come from? "What exactly is the transaction going to turn around? ”. I feel that I have had these problems before, that is, first of all, the concept of tokens is not very clear, and secondly, I can’t understand the virtual _ account . Solve your own lack of knowledge about blockchain and smart contracts. Fortunately, as you look at each module, you can basically understand the overall system architecture and operating principles of NEO. It can be seen that the father of Linux That sentence: "Talk is cheap, show me your code." (don't force it, look at the code) is still very reasonable. Sorry, I force it a bit too much.

To understand the calculation principle of the balance, we must first understand the UTXO transaction model. This thing community leader Li has released a series of explanation videos in the group. If you are interested, you can go to the film. Chapter 6 Transaction of "Mastering BitCoin" also explains this in detail. Students who want to read it can find the download link at the end of the article. The code to calculate the balance is in the wallet class.

Source code location: neo/Wallets/Wallet

public Fixed8 GetBalance(UInt256 asset_id)
{
    return GetCoins(GetAccounts().Select(p => p.ScriptHash))
              .Where(p => !p.State.HasFlag(CoinState.Spent)  //未花费状态
                                  && p.Output.AssetId.Equals(asset_id)) //资产类型
              .Sum(p => p.Output.Value);
}

In fact, it can be clearly seen from here that the calculation process of this balance is to traverse the output corresponding to the address, and add the values ​​that are the same as the specified asset type and whose status is not spent (Spent represents transferred out). A more straightforward explanation is that adding each Output pointing to your account is equivalent to giving you a sum of money, and the sum of all unspent Outputs is your current balance.

0x03 New transaction

In NEO, a Transaction object needs to be constructed when transferring money. This object needs to specify the asset type, transfer amount, asset source (Output), destination address, and witness (Witness). Since the transfer in the GUI can construct a transaction from multiple addresses of the current wallet at the same time, it is more complicated. I will use the code of my applet to explain here. The basic principle is the same.

• ThinSDK-ts：https://github.com/NewEconoLab/neo-thinsdk-ts
• NEOWalletForWeChat：https://github.com/NewEconoLab/NeoWalletForWeChat

The code of the light wallet for transaction processing is modified from NEL's TS light wallet, but the code about the interface in the original code of TS is removed, and then repackaged as a js module, which is equivalent to the GUI transfer function after the devil slimmed down version. Hell-level slimming, greatly reducing the amount of code. The entry of the applet transfer is in the send.wpy interface:

Source code location: neowalletforwechat/src/pages/send.wpy/OnSend

//交易金额
var count = NEL.neo.Fixed8.parse(that.amount + '');
//资产种类
let coin = that.checked === 'GAS' ? CoinTool.id_GAS : CoinTool.id_NEO;
wepy.showLoading({ title: '交易生成中' });
//构造交易对象
var tran = CoinTool.makeTran(UTXO.assets, that.targetAddr, coin, count);
// console.log(tran);
//生成交易id 没错是随机数
let randomStr = await Random.getSecureRandom(256);
//添加资产源、资产输出、见证人、签名
const txid = await TransactionTool.setTran(
  tran,
  prikey,
  pubkey,
  randomStr
);

When calling the makeTran method of CoinTool when constructing a transaction object, four parameters need to be passed in, one is OutPuts, the other is the target account, the third is the asset type, and the last is the asset quantity. This method corresponds to neo/Wallets/Wallet.cs/MakeTransaction<T> in neo core. The two functions are basically the same. The initialization code of the transaction object in makeTran is as follows:

        //新建交易对象
        var tran = new NEL.thinneo.TransAction.Transaction();
        //交易类型为合约交易
        tran.type = NEL.thinneo.TransAction.TransactionType.ContractTransaction;
        tran.version = 0;//0 or 1
        tran.extdata = null;
        tran.attributes = [];
        tran.inputs = [];

In the UTXO transaction model, each transaction will have one or more sources of funds, that is, those Outputs pointing to the current address, and these OutPuts are used as the input of new transactions:

        //交易输入
        for (var i = 0; i < us.length; i++) {
            //构造新的input
            var input = new NEL.thinneo.TransAction.TransactionInput();
            //新交易的prehash指向output
            input.hash = NEL.helper.UintHelper.hexToBytes(us[i].txid).reverse();
            input.index = us[i].n;
            input["_addr"] = us[i].addr;
            //向交易资产来源中添加新的input
            tran.inputs.push(input);
            //计算已添加的资产总量
            count = count.add(us[i].count);
            scraddr = us[i].addr;
            //如果已添加的资产数量大于或等于需要的数量，则不再添加新的
            if (count.compareTo(sendcount) > 0) {
                break;
            }
        }

In a transaction, the output of your own account becomes the input of the new transaction, and then the new transaction will specify a new output. Usually, in addition to an output pointing to the destination account, a transaction will also have an Output for making change. For convenience, I'll tell you a little story here.

• Act 1: Xiaoming writes a blog and sends it to the community. Every time he writes a blog, the community will give Xiaoming 5 GAS as a reward. Every time the community gives Xiaoming a reward, Xiaoming will have an output of 5GAS in his account. Now Xiaoming has written three blogs, and the community has rewarded Xiaoming three times. So Xiao Ming has three outputs of 5GAS.
• Act 2: Xiao Ming hopes to use the GAS earned from blogging to buy cosmetics for his girlfriend. It is known that cosmetics that support GAS payment are priced at 6.8 GAS.
• Act 3: One Output only has 5 GAS, which is obviously not enough to pay for cosmetics, so Xiao Ming had to take out two Outputs to pay.
• Act 4: Since each output is inseparable, just like 100 yuan, you can't tear up a 100 yuan to pay proportionally, you can only give it 100, and they will give you change . The same is true for Output. If you take out two Outputs to pay, then the transaction cannot deduct 1.8 from the existing Output. The only way is to completely set both Outputs to Spent at the same time, then give the merchant an OutPut of 6.8, and then give Xiao Ming an Output of 3.2 as change. Now Xiao Ming has only one 5GAS output and one 3.2GAS output.
• Act 5: Xiaoming bought cosmetics for his girlfriend through his own efforts, and his girlfriend was very happy, so she bought a new washboard for Xiaoming.

UTXO transactions are actually like this. The output cannot be divided. As long as it is transferred out, it will be transferred out together, and then transferred to a new output as change. The code for output construction is as follows:

//输出
if (sendcount.compareTo(NEL.neo.Fixed8.Zero) > 0) {
    var output = new NEL.thinneo.TransAction.TransactionOutput();
    //资产类型
    output.assetId = NEL.helper.UintHelper.hexToBytes(assetid).reverse();
    //交易金额
    output.value = sendcount;
    //目的账户
    output.toAddress = NEL.helper.Helper.GetPublicKeyScriptHash_FromAddress(targetaddr);
    //添加转账交易
    tran.outputs.push(output);
}

//找零
var change = count.subtract(sendcount); //计算找零的额度
if (change.compareTo(NEL.neo.Fixed8.Zero) > 0) {
    var outputchange = new NEL.thinneo.TransAction.TransactionOutput();
    //找零地址设置为自己
    outputchange.toAddress = NEL.helper.Helper.GetPublicKeyScriptHash_FromAddress(scraddr);
    //设置找零额度
    outputchange.value = change;
    //找零资产类型
    outputchange.assetId = NEL.helper.UintHelper.hexToBytes(assetid).reverse();
    //添加找零交易
    tran.outputs.push(outputchange);
}

The above is the process of constructing a new transaction. Basically, the structure of a transaction has been completed. Where it comes from, where it goes, and how much it transfers, it has been constructed, and then the transaction needs to be signed.

0x04 Signature

Unlike traditional face-to-face transactions, how does a transaction published in the network confirm the user's identity? As long as you know the address of the other party, you can get the output of the other party. If only one transfer object can successfully transfer the funds of the other party's account, then this is not all messed up. Therefore, for a transaction, in addition to the elements necessary to construct the transaction, it is also necessary to sign the transaction to prove to the blockchain that the owner of the account is in and out of the transaction. The signature method in NEO Core is defined in the neo/Cryptography/ECC/ECDsa.cs file. Since this part belongs to the category of cryptography and does not belong to the part I want to analyze, I will briefly mention it here:

Source code location: thinsdk-ts/thinneo/Helper.cs/Sign

 //计算公钥
 var PublicKey = ECPoint.multiply(ECCurve.secp256r1.G, privateKey);
 var pubkey = PublicKey.encodePoint(false).subarray(1, 64);
 //获取CryptoKey
 var key = new CryptoKey.ECDsaCryptoKey(PublicKey, privateKey);
 var ecdsa = new ECDsa(key);
 {
     //签名
     return new Uint8Array(ecdsa.sign(message,randomStr));
 }

The real signature part is actually the standard ECDsa digital signature. The return value is a Uint8 array of length 64, the first 32 bytes are R, and the last 32 bytes are S:

let arr = new Uint8Array(64);
Arrayhelper.copy(r.toUint8Array(false, 32), 0, arr, 0, 32);
Arrayhelper.copy(s.toUint8Array(false, 32), 0, arr, 32, 32);
return arr.buffer;

Parameters S and R are very important parameters in ECDsa digital signature verification.

0x05 Witness

It is not enough to just calculate the signature, we also need to add the signature to the transaction. This process is to add a witness:

tran.AddWitness(signdata, pubkey, WalletHelper.wallet.address);

The process of adding a witness is actually to push the signature information of the previous step and the verification information obtained through the public key into the witness script. The process of adding a witness without the complex verification process is as follows:

Source code location: thinsdk-ts/thinneo/Transaction.cs

//增加个人账户见证人（就是用这个人的私钥对交易签个名，signdata传进来）
 public AddWitness(signdata: Uint8Array, pubkey: Uint8Array, addrs: string): void {
    var vscript = Helper.GetAddressCheckScriptFromPublicKey(pubkey);
    //iscript 对个人账户见证人他是一条pushbytes 指令
    var sb = new ScriptBuilder();
    sb.EmitPushBytes(signdata);
    var iscript = sb.ToArray();
    this.AddWitnessScript(vscript, iscript);
}

//增加智能合约见证人
public AddWitnessScript(vscript: Uint8Array, iscript: Uint8Array): void {
    var newwit = new Witness();
    newwit.VerificationScript = vscript;
    newwit.InvocationScript = iscript;
    //添加新见证人
    this.witnesses.push(newwit);

}

I still have a problem here, that is, this transaction involves multiple accounts under one wallet. Should there be multiple signatures? Theoretically yes, after all, each account has its own independent private key, so I looked at the transfer code of the GUI again. The entry for the GUI to obtain the transaction and sign is the transfer method located in the MainForm file, which calls the Helper In the SignAndShowInformation, in this SignAndShowInformation, the Sign method in the Wallet file is called, and the transaction context is passed in:

Source code location: neo/Wallets/Wallet.cs

public bool Sign(ContractParametersContext context)
{
    bool fSuccess = false;
    foreach(UInt160 scriptHash in context.ScriptHashes)
    {
        WalletAccount account = GetAccount(scriptHash);
        if (account ?.HasKey != true) continue;
        KeyPair key = account.GetKey();
        byte[] signature = context.Verifiable.Sign(key);
        fSuccess |= context.AddSignature(account.Contract, key.PublicKey, signature);
    }
    return fSuccess;
}

As can be seen from the source code, NEO will use the private key of each account participating in the transaction to sign the transaction. After the signature is completed, the GetScripts method is called to get the script, and it is in this method that the transaction adds multiple witnesses:

public Witness[] GetScripts()
{
    if (!Completed) throw new InvalidOperationException();
    // 脚本哈希数量 == 见证人数量
    Witness[] scripts = new Witness[ScriptHashes.Count];
    for (int i = 0; i < ScriptHashes.Count; i++)
    {
        ContextItem item = ContextItems[ScriptHashes[i]];
        using(ScriptBuilder sb = new ScriptBuilder())
        {
            foreach(ContractParameter parameter in item.Parameters.Reverse())
            {
                sb.EmitPush(parameter);
            }
            //添加新的见证人
            scripts[i] = new Witness
            {
                InvocationScript = sb.ToArray(),
                    VerificationScript = item.Script ?? new byte[0]
             };
        }
    }
    return scripts;
}

Therefore, if your GUI wallet transfers an amount larger than the amount of a single account, it is actually a transaction completed by multiple accounts.

Download link: "Mastering BitCoin": https://github.com/Liaojinghui/BlockChainBooks

{{o.name}}

{{m.name}}