foreword
This article mainly introduces the underlying protocol stack of IPFS, including the seven-layer structure of IPFS.
1. Overview of IPFS protocol
Similar to HTTP, IPFS is an application layer protocol based on TCP/IP. The IPFS protocol stack is also composed of seven layers of sub-protocols responsible for different functions. As shown in the figure below:
1. Identity layer: management node identity generation and verification.
2. Network layer: manages connections with other nodes.
3. Routing layer: DHT maintains routing information to locate specific peer nodes and objects, and responds to query requests from local and remote nodes.
4. Exchange layer: Based on BitSwap (Bit Exchange Protocol), it simulates a trusted market, weakens data duplication, and prevents cheating.
5. Object layer: Based on Merkle DAG, it has the characteristics of content addressing and anti-redundancy.
6. File layer: Similar to the version file system of Git, it supports blob, list, tree and other structures.
7. Naming layer: a variable name system with self-inspection characteristics.
Then, I will introduce the seven-layer structure one by one below:
2. Identity layer
In the IPFS network, all nodes are identified by a unique NodeId. Similar to a Bitcoin address, NodeId is also a hash of a public key. Each node is represented by a Node structure in the IPFS code, which only contains NodeId and a set of public-private key pairs.
type NodeId Multihash
type Multihash []byte // 自描述加密哈希摘要
type PublicKey []byte
type PrivateKey []byte // 自描述的私钥
type Node struct {
NodeId NodeID
PubKey PublicKey
PriKey PrivateKey
}
The main function of the identity system is to identify nodes in the IPFS network. Similar to the generation of user information. When the nodes establish a connection for the first time, the nodes first exchange public keys and perform identity verification.
3. Network layer
The IPFS network stack has the following characteristics:
1. Transmission: IPFS is compatible with existing mainstream transmission protocols, including WebRTC DataChannels, uTP and other transmission protocols.
2. Reliability: use uTP and sctp to guarantee and dynamically adjust the network status.
3. Connectivity: Use ICE to achieve WAN connectivity.
4. Integrity: Check data integrity using hash checksum. All data blocks in the IPFS network have a unique hash value.
5. Verifiability: Use the public key of the data sender and the HMAC message authentication code to check the authenticity of the message.
4. Routing layer
The IPFS routing layer data structure uses a distributed loose hash table (DSHT) based on S/Kademlia and Coral technology , which is mainly used to implement three basic functions: content routing, node routing, and data storage. The DHT structure of IPFS will be distinguished according to the size of the stored data: small values are directly stored on the DHT, generally no more than 1KB; for larger values, the DHT only stores the value index, which is a NodeId.
5. Exchange layer
The exchange layer of IPFS uses the BitSwap protocol, and its main function is to use the credit mechanism to exchange data between nodes, and each node continuously uploads the downloaded data to other nodes while downloading. In IPFS, the distribution and exchange of data use the BitSwap protocol.
The BitSwap protocol is mainly responsible for two things:
1. Request a list of required data blocks from other nodes
2. Provide other nodes with a list of existing data blocks.
The life cycle of BitSwap data exchange goes through 4 states:
1. State development (Open): peer nodes develop the status of BitSwap bills to be sent until the connection is established.
2. Data sending (Sending): sending want_list and data blocks between nodes.
3. Connection close (Close): The node disconnects after sending the data.
4. Node Ignoring: Nodes are ignored due to factors such as timeout, customization, and low credit score.
6. Object layer
The IPFS object layer mainly uses Merkle DAG technology to construct a directed acyclic graph data structure to store object data.
Merkle DAG provides some useful properties for IPFS, including:
1. Content addressing: All content is verified and uniquely identified by multiple hashes.
2. Prevent tampering: If data is tampered with or damaged in the IPFS network, it can be detected through hash verification.
3. Deduplication: All objects holding the exact same content are identical and stored only once.
Seven, file layer
IPFS also defines a set of objects for modeling versioned filesystems on top of Merkle DAGs.
1. Block: A data block of variable size.
2. List (list): A collection of blocks or other lists.
3. tree: a collection of blocks, lists, or other trees
4. commit: a snapshot in the tree version history.
8. Naming layer
At the naming layer, the Protocol Labs team designed the IPNS interplanetary file command system module for IPFS.
Among them, the method of self-validating naming is adopted. The mode is as follows:
1. Generate IPFS node information through NodeId =hash(node.Pubkey)
2. Assign each user a variable namespace, using the previously generated node ID information as the address name, under this path :/ipns/.
3. A user can publish an object signed with his own private key under this path.
For example: /ipns/XLF2ipQ4jD3UdeX5xp1KBgeHRhemUtaA8Vm/
4. When other users obtain the object, they can check whether the signature matches the public key and node information, thereby verifying the authenticity of the object published by the user.
In addition, IPFS also uses some technologies to increase the user-friendliness of IPNS.
1. Peer node link
Users can directly link objects of other user nodes to their own namespace.
2.DNS TXT IPNS record
Users can add TXT records to the existing DNS system, so that they can access file objects in the IPFS network through domain names.
#DNS TXT record
ipfs.benet.ai.TXT “ipfs=XLF2ipQ4jD3U …”
#Represented as a symbolic link
ln -s /ipns/XLF2ipQ4jD3U /ipns/fs.benet.ai
IPFS also supports the readable identifier Proquint, which can translate binary codes into readable
files
#proquint statement
/ipns/dahih-dolij-sozuk-vosah-luvar-fuluh
#Decomposed into the corresponding form
/ipns/KhAwNprxYVxKqpDZ
In addition, IPFS also provides short address naming services
#Users can get a link from below
/ipns/shorten.er/foobar
#Then put it in their own namespace
/ipns/XLF2ipQ4jD3UdeX5xp1KBgeHRhemUtaA8Vm
Summarize
The above is the IPFS protocol stack to be talked about today, including the seven-layer structure.