foreword
As we all know, to develop EOS robot and WAX chain game scripts, we all need to call eosio chain api:
https://developers.eos.io/manuals/eos/latest/nodeos/plugins/chain_api_plugin/api-reference/index
It can be seen that it is all based on the HTTP protocol. In theory, we can directly use the HTTP client to interact with it, such as the [requests] package in python, but for querying data, it is acceptable to send HTTP requests directly. But when submitting a transaction, it involves parsing chain info, packaging the transaction and signing, and it is too troublesome to invent the wheel from scratch, so we generally use eosio sdk to do these things.
Unfortunately, eosio does not officially provide an sdk based on the python language. The most mainstream sdk officially supported is based on javascript
[eosjs]: https://github.com/EOSIO/eosjs
As for the sdk based on python, we mainly used [eospy] developed by third-party individual developers before
: https://github.com/eosnewyork/eospy
[pyeoskit]: https://github.com/learnforpractice/pyeoskit
There are also some niche ones:
[ueosio] https://github.com/EOSArgentina/ueosio (too low-level, troublesome to use)
[eosjs_python] https://github.com/EvaCoop/eosjs_python (using python and nodejs The interaction ends up calling eosjs...)
However, in the process of developing automated scripting tools for WAX chain games, after in-depth use, we found some problems. The above tools still cannot meet our needs very well.
【eospy】Existing problems:
1. The details of the internal HTTP library are shielded, and it is inconvenient to modify the parameters of the HTTP request.
[eospy] internally uses [requests] to initiate HTTP requests, but does not expose the requests object interface. If we need to modify the parameters of HTTP requests, it will be more troublesome, such as setting http/socks5 proxy, modifying http timeout value, modifying http Request header (such as "User-Agent").
In addition, [requests] will use the system HTTP proxy settings by default. If your computer has a tool such as a ladder, [eospy] HTTP requests will go through the system proxy when initiating a transaction, resulting in some unexpected effects. Of course, we can modify the session.trust_env = False of [requests] to prevent it from using the system proxy.
Regarding the appeal, we can directly modify the source code of [eospy] to meet our needs, but the effect of such tinkering and modification is not very comfortable.
2. There is no perfect session isolation mechanism
What does session isolation mean? We know that the requests package can use requests.Session() to create an object. Multiple session objects do not interfere with each other. You can set different proxies, different http request headers, and different timeouts value and other parameters. Similarly, our ideal eosio sdk can also instantiate a session object, and each session object can set different agents, bind different accounts and private keys, and use different rpc nodes.
This is very important when developing automatic robots and chain game scripts, because a program often runs dozens or hundreds of accounts at the same time, and each account must be set with a different proxy IP, or even use a different rpc node.
ce = eospy.cleos.Cleos(url="https://jungle3.greymass.com")
[eospy] Although session objects can be instantiated, the next two steps need to send HTTP requests: ce.abi_json_to_bin and ce.push_transaction, multiple session objects use the same http configuration, ce.push_transaction is okay, proxies are provided parameters and timeout parameters, but ce.abi_json_to_bin does not provide the proxies parameter. Of course, these small problems can be solved by simply modifying [eospy].
3. Error handling
When an eosio automatic robot or blockchain game script is running, there are often three types of errors.
① One is a network error, such as network timeout, network interruption, proxy problem
② One is a node error, eos/wax node denial of service, such as a node failure, or too frequent access that returns http 429 or http 403 denial of
service③ One is a transaction error, that is, the transaction has been submitted, but because of insufficient CPU, insufficient balance, signature error, permission problem, contract error, etc., an http 500 error is returned
These three types of errors should be clearly distinguished and handled in the blockchain game script. If they are network errors, they can be retried on the spot. If they are node errors, adjust the access frequency or switch agents. They are transaction errors and should not be retried blindly.
However, in [eospy], for the above three errors, [eospy] internally throws the http exception of the underlying [requests] library, and does not clearly distinguish the error category, which is a bit cumbersome and not very elegant to handle.
4. The transaction is not serialized
Let's use [eospy] to initiate a simple transaction:
import datetime
import eospy.cleos
import eospy.keys
import pytz
consumer_name = "consumer1111"
consumer_private_key = eospy.keys.EOSKey("5KWxgG4rPEXzHnRBaiVRCCE6WAfnqkRpTu1uHzJoQRzixqBB1k3")
ce = eospy.cleos.Cleos(url="https://jungle3.greymass.com")
def main():
action = {
"account": 'eosio.token',
"name": 'transfer',
"authorization": [
{
"actor": consumer_name,
"permission": "active",
},
],
"data": {
"from": consumer_name,
"to": "consumer2222",
"quantity": "0.0001 EOS",
"memo": "by eospy",
},
}
data = ce.abi_json_to_bin(action['account'], action['name'], action["data"])
action["data"] = data["binargs"]
tx = {
"actions": [action],
"expiration": str((datetime.datetime.utcnow() + datetime.timedelta(seconds=90)).replace(tzinfo=pytz.UTC))
}
resp = ce.push_transaction(tx, consumer_private_key)
print(resp)
if __name__ == '__main__':
main()
Through debugging or packet capture, we can see that the final HTTP request body sent by [eospy] is:
{
"compression":"none",
"transaction":{
"expiration":"2022-05-25T16:09:46.186449+00:00",
"ref_block_num":4469,
"ref_block_prefix":4235931364,
"net_usage_words":0,
"max_cpu_usage_ms":0,
"delay_sec":0,
"context_free_actions":[],
"actions":[
{
"account":"eosio.token",
"name":"transfer",
"authorization":[
{
"actor":"consumer1111",
"permission":"active"
}
],
"data":"10420857498d274520841057498d2745010000000000000004454f530000000008627920656f737079"
}
],
"transaction_extensions":[]
},
"signatures":[
"SIG_K1_Kei6azGcSWP61M5uVNU7s7HAizGnrP4Q9BA6j557XVmeWsFKGEkdNv1QtaHAP7JKzhCSRFaxq5HbX3dqkzVzMraKtnoLT3"
]
}
Next use [eosjs] to send the same transaction:
async function transfer() {
const consumer_name = "consumer1111";
const consumer_private_key = "5KWxgG4rPEXzHnRBaiVRCCE6WAfnqkRpTu1uHzJoQRzixqBB1k3";
const rpc = new eosjs_jsonrpc.JsonRpc("https://jungle3.greymass.com");
const provider = new eosjs_jssig.JsSignatureProvider([consumer_private_key]);
const api = new eosjs_api.Api({
rpc:rpc, signatureProvider: provider });
const result = await api.transact({
actions: [{
account: 'eosio.token',
name: 'transfer',
authorization: [
{
actor: consumer_name,
permission: "active",
},
],
data: {
from: consumer_name,
to: "consumer2222",
quantity: '0.0001 EOS',
memo: 'by eosjs',
},
}]
}, {
blocksBehind: 3,
expireSeconds: 90,
});
console.log(result)
}
Through debugging or packet capture, we can see that the final HTTP request body sent by [eosjs] is:
{
"signatures":[
"SIG_K1_KkKSHRez98XBv6EyQhgmiLtRUbM5WKTb72iHUFK7K9zf4cMHFDZQi8Kd4vttRHxjRYseMo1kQa7vKvvKbojkJHrqCF12bK"
],
"compression":0,
"packed_context_free_data":"",
"packed_trx":"00588e621619c0bc13a1000000000100a6823403ea3055000000572d3ccdcd0110420857498d274500000000a8ed32322910420857498d274520841057498d2745010000000000000004454f530000000008627920656f736a7300"
}
You can see the difference, [eospy] does not pack and serialize the transaction body, while [eosjs] serializes the transaction into binary data before sending it.
Although these two methods can be accepted by many eos or wax node rpc servers, [eospy] is obviously an outdated method and has not been updated in time.
We can check the latest eosio rpc documentation to confirm this:
https://developers.eos.io/manuals/eos/latest/nodeos/plugins/chain_api_plugin/api-reference/index#operation/push_transaction
The most recommended way is still Pack it into packed_trx and then send it.
This creates two problems:
1. Some public eos or wax nodes do not support the old data format [eospy], but only support the latest packed_trx method, which makes these nodes unusable.
2. If you do not have a private key locally, you need to sign the transaction through the server, such as Wax cloud wallet, you need to pack the transaction into packed_trx, post it to the Wax cloud wallet server for signature, and then push it to the Wax network. [eospy] If it does not support packing into packed_trx, it will be more troublesome.
In fact, if you study the source code of [eospy] carefully, you will find that [eospy] itself has already provided the serialization function required for the packed transaction packed_trx. To solve this problem, we can also modify the source code of [eospy], but it is modified. Code is not always elegant.
【pyeoskit】Existing problems:
Sample code:
from pyeoskit import eosapi, wallet
consumer_name = "consumer1111"
consumer_private_key = "5KWxgG4rPEXzHnRBaiVRCCE6WAfnqkRpTu1uHzJoQRzixqBB1k3"
wallet.import_key(consumer_name, consumer_private_key)
eosapi.set_node("https://jungle3.greymass.com")
def main():
data = {
"from": consumer_name,
"to": "consumer2222",
"quantity": "0.0001 EOS",
"memo": "by pyeoskit",
}
authorization = {
consumer_name: "active"
}
action = ["eosio.token", "transfer", data, authorization]
resp = eosapi.push_action(*action)
print(resp)
if __name__ == '__main__':
main()
[pyeoskit] solves many pain points of [eospy], it can pack the transaction into packed_trx and send it, but it brings new problems:
-
[pyeoskit] is not a library implemented by pure python. It uses golang at the bottom layer to realize the serialization and signature of transactions, and then compiles it into native code for python calls. Its python code part is just a small layer of skin. Although this can improve Operating efficiency, especially for binary data processing, golang implementation will be faster. But after relying on golang, if we need to patch the code at any time, it will be troublesome.
-
Trust issues, because the package released by [pyeoskit] on [pypi] contains compiled binary executable files (the part implemented by golang), whether this part is consistent with the open source code on github, or inserts malicious code, we It is not known that when we use it to develop robots and chain game scripts, we often need to directly import private keys, so we must attach great importance to security. Of course, we can download his golang code from github to compile, but it is a little troublesome.
-
Compatibility, I don't know how the author of the project compiled it, but the package he released on [pypi], some versions of it, will crash when running on my win10 computer.
-
There are many bugs. After using it in depth, you will find that this library has many bugs. We have submitted several issues to fix it, and the library is quite chaotic. It uses three sets of HTTP libraries at the same time, [requests] [httpx] and golang The built-in http library makes it very confusing when sending transactions. [httpx] is used for abi_json_to_bin, and [requests] is used for push_transaction, which is easy to step on.
-
There are also three problems of [eospy] 1, 2, and 3.
However, [pyeoskit] also has some advantages:
-
It supports serialization of data from abi files, and built-in some abi files commonly used by eosio, so that when packaging transactions, there is no need to send abi_json_to_bin request to serialize data data, which reduces the http request pressure on eos nodes.
-
It supports abi cache. For [pyeoskit] smart contracts that do not have a built-in abi file, such as [farmersworld] in the farmer's world, it will get the abi of the contract through /v1/chain/get_abi when calling the contract for the first time and save it in memory , in the next repeated calls, there is no need to call abi_json_to_bin to serialize data, which reduces the http request pressure on the eos node.
-
It has a very complete function. For example, before sending a transaction, it will call get_required_keys to check the keys required to sign the transaction and whether the local key meets the conditions, so as to avoid sending transactions with incomplete signatures to the server, and do a good job of checking locally. Throw an exception. Another example is that it implements the compression of push_transaction, that is, further compresses the packed_trx serialized into binary data to reduce the transmission volume.
Generally speaking, [pyeoskit] is still an excellent and "modern" eosio sdk. After all, it was only released in the last year. Compared with old antiques like [eospy], the code structure and design concept are more advanced. The only pity is that it is basically written in golang, and then wrapped in python. If it is written in pure python, it will be perfect. As for the problem of many bugs, it is mainly caused by few users, low attention, and imperfect testing. If the popularity rises, I believe that no matter how many bugs there are, they will be fixed soon.