The rlpx protocol of the Ethereum source code analysis

The rlpx protocol of the Ethereum source code analysis

This article mainly refers to the official documentation of eth: rlpx protocol

symbol

X || Y: Indicates the concatenation of X and Y
X ^ Y: X and Y bitwise XOR
X[:N]: the first N bytes of X
[X, Y, Z, ...]: RLP recursive encoding of [X, Y, Z, …]
keccak256(MESSAGE): The keccak256 hash algorithm used by Ethereum
ecies.encrypt(PUBKEY, MESSAGE, AUTHDATA): The asymmetric authentication encryption function AUTHDATA used by RLPx is the authentication data, not part of the ciphertext, but AUTHDATA will be written into the HMAC-256 hash function before generating the message tag
ecdh.agree(PRIVKEY, PUBKEY): is the elliptic curve Diffie-Hellman negotiation function between PRIVKEY and PUBKEY

ECIES encryption

ECIES (Elliptic Curve Integrated Encryption Scheme) asymmetric encryption is used for RLPx handshake. The encryption system used by RLPx:

Elliptic curve secp256k1 base pointG
KDF(k, len): Key derivation function NIST SP 800-56 Concatenation
MAC(k, m): HMAC function, using SHA-256 hash
AES(k, iv, m): AES-128 symmetric encryption function, CTR mode

Suppose Alice wants to send an encrypted message to Bob, and hopes that Bob can decrypt it with his static private key kB. Alice knows Bob's static public key KB.

Alice in order to mencrypt the message:

Generate a random number rand generate the corresponding elliptic curve public keyR = r * G
Compute the shared secret S = Px, where(Px, Py) = r * KB
kE || kM = KDF(S, 32)Derivation of keys and random vectors required for encryption and authenticationiv
Encrypt with AESc = AES(kE, iv, m)
Calculate MAC checksumd = MAC(keccak256(kM), iv || c)
Send the full ciphertext R || iv || c || dto Bob

R || iv || c || dBob decrypts the ciphertext :

derive the shared secret S = Px, where(Px, Py) = r * KB = kB * R
Deriving the key for encryption and authenticationkE || kM = KDF(S, 32)
Verify MACd = MAC(keccak256(kM), iv || c)
get plaintextm = AES(kE, iv || c)

node identity

All cryptographic operations are based on the secp256k1 elliptic curve. Each node maintains a static secp256k1 private key. It is recommended that the private key should only be manually reset (eg deleting a file or database entry).

handshake process

RLPx connection is based on TCP communication, and each communication will generate a random ephemeral key for encryption and authentication. The process of generating a temporary key is called "handshake", and the handshake is performed between the initiator (the node that initiates the TCP connection request) and the recipient (the node that accepts the connection).

The initiator initiates a TCP connection to the receiver and sends autha message
Receiver accepts connection, decrypts, verifies authmessage (check recovery of signature == keccak256(ephemeral-pubk))
The receiver generates messages via andremote-ephemeral-pubknonceauth-ack
The receiving end derives the key and sends the first data frame (frame) containing the Hello message
The initiator receives auth-ackthe message and derives the key
The initiator sends the first encrypted data frame, including the initiator Hello message
Receiver receives and verifies the first encrypted data frame
The initiator receives and verifies the first encrypted data frame
If the MAC of the first encrypted data frame on both sides is verified, the encryption handshake is completed

If validation of the first data frame fails, either party can disconnect.

handshake message

sender:

auth = auth-size || enc-auth-body
auth-size = size of enc-auth-body, encoded as a big-endian 16-bit integer
auth-vsn = 4
auth-body = [sig, initiator-pubk, initiator-nonce, auth-vsn, ...]
enc-auth-body = ecies.encrypt(recipient-pubk, auth-body || auth-padding, auth-size)
auth-padding = arbitrary data

Receiving end:

ack = ack-size || enc-ack-body
ack-size = size of enc-ack-body, encoded as a big-endian 16-bit integer
ack-vsn = 4
ack-body = [recipient-ephemeral-pubk, recipient-nonce, ack-vsn, ...]
enc-ack-body = ecies.encrypt(initiator-pubk, ack-body || ack-padding, ack-size)
ack-padding = arbitrary data

Implementations must ignore all mismatches inauth-vsn and .ack-vsn

Implementations must ignore all extra list elements inauth-body and .ack-body

After the handshake messages are exchanged, the key is generated:

static-shared-secret = ecdh.agree(privkey, remote-pubk)
ephemeral-key = ecdh.agree(ephemeral-privkey, remote-ephemeral-pubk)
shared-secret = keccak256(ephemeral-key || keccak256(nonce || initiator-nonce))
aes-secret = keccak256(ephemeral-key || shared-secret)
mac-secret = keccak256(ephemeral-key || aes-secret)

frame structure

After the handshake all messages are transmitted in frames. One frame of data carries one encrypted message belonging to a certain function.

The main purpose of framing is to reliably support multiplexing protocols over a single connection. Second, encryption streams are simplified due to packet framing, which creates appropriate demarcation points for message authentication codes. The data frame is encrypted and authenticated with the key generated by the handshake.

The frame header provides information about the size of the message and the capabilities of the source of the message. Padding bytes are used to prevent buffer starvation so that frame components are aligned to the specified block byte size.

frame = header-ciphertext || header-mac || frame-ciphertext || frame-mac
header-ciphertext = aes(aes-secret, header)
header = frame-size || header-data || header-padding
header-data = [capability-id, context-id]
capability-id = integer, always zero
context-id = integer, always zero
header-padding = zero-fill header to 16-byte boundary
frame-ciphertext = aes(aes-secret, frame-data || frame-padding)
frame-padding = zero-fill frame-data to 16-byte boundary

MAC

Message authentication in RLPx uses two keccak256 states for two transmission directions respectively. egress-macand ingress-macrepresent the sending and receiving status respectively, and the status will be updated every time the ciphertext is sent or received. After the initial handshake, the MAC state is initialized as follows:

sender:

egress-mac = keccak256.init((mac-secret ^ recipient-nonce) || auth)
ingress-mac = keccak256.init((mac-secret ^ initiator-nonce) || ack)

Receiving end:

egress-mac = keccak256.init((mac-secret ^ initiator-nonce) || ack)
ingress-mac = keccak256.init((mac-secret ^ recipient-nonce) || auth)

When sending a frame of data, the state is updated by the data to be sent egress-mac, and then the corresponding MAC value is calculated. Updates are made by XORing the encrypted output of the frame header with its corresponding MAC value. This is done to ensure uniform operations on plaintext MACs and ciphertexts. All MAC values are sent in clear text.

header-mac-seed = aes(mac-secret, keccak256.digest(egress-mac)[:16]) ^ header-ciphertext
egress-mac = keccak256.update(egress-mac, header-mac-seed)
header-mac = keccak256.digest(egress-mac)[:16]

calculateframe-mac

egress-mac = keccak256.update(egress-mac, frame-ciphertext)
frame-mac-seed = aes(mac-secret, keccak256.digest(egress-mac)[:16]) ^ keccak256.digest(egress-mac)[:16]
egress-mac = keccak256.update(egress-mac, frame-mac-seed)
frame-mac = keccak256.digest(egress-mac)[:16]

The MAC value in the ingress frame can be verified as long as the sender and receiver update the sum in the same way egress-macand compare the values of and in the ingress frame. This step should be done before decrypting and .ingress-macheader-macframe-macheader-ciphertextframe-ciphertext

function message

All messages after the initial handshake are "capabilities" related. Any number of functions can be used simultaneously on a single RLPx connection.

Features are identified by a short ASCII name and a version number. Capabilities supported by both ends of the connection are exchanged in Hello messages belonging to the "p2p" capability that needs to be available in all connections.

message encoding

The initial Hello message is encoded as follows:

frame-data = msg-id || msg-data
frame-size = length of frame-data, encoded as a 24bit big-endian integer

where msg-idis the RLP-encoded integer identifying the message and msg-datais the RLP list containing the message data.

All messages after Hello are compressed using the Snappy algorithm. Note that the size of a compressed message frame-sizerefers to msg-datathe size before compression. The compressed encoding of the message is:

frame-data = msg-id || snappyCompress(msg-data)
frame-size = length of (msg-id || msg-data) encoded as a 24bit big-endian integer

Based `msg-id`on multiplexing

Although it is supported in the frame capability-id, this field is not used for multiplexing between different functions in this RLPx version (the current version only uses msg-id to achieve multiplexing).

Each function allocates as much msg-id space as needed. The msg-id space required by all these functions must be specified via static. When connecting and receiving Hello messages, both ends have peer information on shared capabilities (including versions), and are able to reach consensus on the msg-id space.

msg-id should be greater than 0x11 (0x00-0x10 are reserved for "p2p" function).

p2p function

All connections are "p2p" capable. After the initial handshake, both ends of the connection must send Hello or Disconnect messages. After the Hello message is received, the session becomes active and can start sending other messages. Implementations MUST ignore all differences in protocol versions due to forward compatibility. When communicating with nodes at lower versions, implementations should try to stay close to that version.

A Disconnect message may be received at any time .

Hello (0x00)

[protocolVersion: P, clientId: B, capabilities, listenPort: P, nodeKey: B_64, ...]

After the handshake is completed, the first packet of data sent by both parties. No other messages can be sent until the Hello message is received. Implementations MUST ignore all other list elements in the Hello message, as they may be used in a future release.

protocolVersionThe current p2p function version is version 5
clientIdRepresents the client software identity, a human-readable string, such as "Ethereum(++)/1.0.0"
capabilitiesList of supported subprotocols, their names and their versions:[[cap1, capVersion1], [cap2, capVersion2], ...]
listenPortThe node's listening port (the interface on the current connection path), 0 means no listening
nodeIdThe public key of secp256k1 corresponds to the private key of the node

Disconnect (0x01)

[reason: P]

Notify the node to disconnect. After receiving this message, the node should immediately disconnect. If it is sending, the normal host will give the node 2 seconds to read, so that it will actively disconnect.

reasonAn optional integer indicating the reason for disconnection:

Reason	Meaning
`0x00`	Disconnect requested
`0x01`	TCP sub-system error
`0x02`	Breach of protocol, e.g. a malformed message, bad RLP, …
`0x03`	Useless peer
`0x04`	Too many peers
`0x05`	Already connected
`0x06`	Incompatible P2P protocol version
`0x07`	Null node identity received - this is automatically invalid
`0x08`	Client quitting
`0x09`	Unexpected identity in handshake
`0x0a`	Identity is the same as this node (i.e. connected to itself)
`0x0b`	Ping timeout
`0x10`	Some other reason specific to a subprotocol

Ping (0x02)

[]

Ask the node for an immediate Pong reply.

Pong (0x03)

[]

Reply to the node's Ping packet.

Source code analysis

The main function

return transfer object

Returns a transport object, the connection lasts for 5 seconds

// handshakeTimeout 5
func newRLPX(fd net.Conn) transport {
    
    
....
}

read message

Return the Msg object, call ReadMsg of the reader, and the connection lasts for 30 seconds

func (t *rlpx) ReadMsg() (Msg, error) {
    
    
  ..
	t.fd.SetReadDeadline(time.Now().Add(frameReadTimeout))
}

write message

Call WriteMsg of the reader to write information, and the connection lasts for 20 seconds

func (t *rlpx) WriteMsg(msg Msg) error {
    
    
  ...
	t.fd.SetWriteDeadline(time.Now().Add(frameWriteTimeout))
}

Protocol version handshake

Protocol handshake, the input and output are both protoHandshake objects, including the version number, name, capacity, port number, ID and an extended attribute, which will be verified during the handshake

encrypted handshake

The initiator of the handshake is called the initiator

The receiver is called receiver

Corresponding to two processing methods initiatorEncHandshake and receiverEncHandshake respectively

After the two processing methods are successful, a secrets object will be obtained, which saves the shared key information, and it will generate a frame processor together with the original net.Conn object: rlpxFrameRW

The information used by both parties in the handshake includes: their respective public-private key address pairs** (iPrv, iPub, rPrv, rPub) , their respective generated random public-private key pairs (iRandPrv, iRandPub, rRandPrv, rRandPub) , and their respective generated temporary random numbers ( initNonce, respNonce).**
where i starts with the initiator** (initiator) information, r starts with the receiver (receiver)** information.

func (t *rlpx) doEncHandshake(prv *ecdsa.PrivateKey, dial *ecdsa.PublicKey) (*ecdsa.PublicKey, error) {
    
    
	var (
		sec secrets
		err error
	)
	if dial == nil {
    
    
		sec, err = receiverEncHandshake(t.fd, prv) // 接收者
	} else {
    
    
		sec, err = initiatorEncHandshake(t.fd, prv, dial) //主动发起者
	}
...
	t.rw = newRLPXFrameRW(t.fd, sec)
	t.wmu.Unlock()
	return sec.Remote.ExportECDSA(), nil
}

Here we explain the source code of the active handshake part initiatorEncHandshake:

①: Initialize the handshake object

h := &encHandshake{initiator: true, remote: ecies.ImportECDSAPublic(remote)}

②: Generate verification information

authMsg, err := h.makeAuthMsg(prv)

func (h *encHandshake) makeAuthMsg(prv *ecdsa.PrivateKey) (*authMsgV4, error) {
	// 生成己方随机数initNonce
	h.initNonce = make([]byte, shaLen)
	_, err := rand.Read(h.initNonce)
...
	}
// 生成随机的一组公私钥对
	h.randomPrivKey, err = ecies.GenerateKey(rand.Reader, crypto.S256(), nil)
...
	}
	// 生成静态共享秘密token(用己方私钥和对方公钥进行有限域乘法)
	token, err := h.staticSharedSecret(prv)
	...
	}
//  和己方随机数异或后用随机生成的私钥签名
	signed := xor(token, h.initNonce)
	signature, err := crypto.Sign(signed, h.randomPrivKey.ExportECDSA())
...
	}
...
	return msg, nil
}

③: Encapsulation, rlp encoding the verification information and handshake and splicing prefix information

authPacket, err := sealEIP8(authMsg, h)

④: Send a message through conn

conn.Write(authPacket)

⑤: Process the received information and get the response packet

readHandshakeMsgeasier. Try decoding with one format first. If not, try another one. Should be a compatibility setting. Basically, use your own private key to decode and then call rlp to decode into a structure.

The description of the structure is the following authRespV4, the most important of which is the random public key of the peer. Both parties can obtain the same shared secret through their own private key and the random public key of the opposite end. And this shared secret is not available to third parties

	authRespMsg := new(authRespV4)
	authRespPacket, err := readHandshakeMsg(authRespMsg, encAuthRespLen, prv, conn)

⑥: Fill in the response's respNonce (the other party's random number, used to generate the shared private key) and remoteRandomPub (the other party's random public key)

 h.handleAuthResp(authRespMsg)

⑦: Encapsulate the request packet and response packet into shared secrets (secrets)

h.secrets(authPacket, authRespPacket)

At this point, the more important content related to RLPX is almost explained.

reference

https://github.com/blockchainGuide/blockchainguide ☆ ☆ ☆ ☆ ☆

https://mindcarver.cn/ ☆ ☆ ☆ ☆ ☆

https://github.com/ethereum/devp2p/blob/master/rlpx.md