Article Directory
Data transmission at the application layer needs to notify the tcp transmission. After the TCP connection has been established, send data to the other party through the tcp_write() function.
1. Introduction
tcp_write() sends data to the other party through the tcp control block of the established connection. The implementation logic of the code is to copy the data to the unsent queue of the control block. In the code, to save memory, the sending data is connected to the unsent in various situations.
Second, code analysis
When sending data into the unsent queue, three situations need to be considered:
1. Write the last pbuf
Since tcp is a byte stream-oriented transmission protocol, the last pbuf in the unsent queue is filled with data if there is remaining memory available.
u16_t space; //报文可用的内存空间，是一个抽象的数值
u16_t unsent_optlen; //选项长度
//找到unsent队列最后一个成员
for (last_unsent = pcb->unsent; last_unsent->next != NULL;
last_unsent = last_unsent->next);
unsent_optlen = LWIP_TCP_OPT_LENGTH(last_unsent->flags); //最后一个报文的选项长度
LWIP_ASSERT("mss_local is too small", mss_local >= last_unsent->len + unsent_optlen);
//last_unsent的报文剩余内存 = 本地最大报文长度-（last_unsent里tcp数据长度+选项长度）
space = mss_local - (last_unsent->len + unsent_optlen);
//将数据复制到最后一个pbuf的剩余内存中，这里并不直接复制，而是记录复制所需的参数（oversize_used）
oversize = pcb->unsent_oversize; //最后一个pbuf剩下的空间，是一个真实的内存
//如果最后的pbuf中还有剩余内存
if (oversize > 0) {
LWIP_ASSERT("inconsistent oversize vs. space", oversize <= space);
seg = last_unsent;
//理解space，oversize大小可能不同，取其中的最小值，len一般都比较大
oversize_used = LWIP_MIN(space, LWIP_MIN(oversize, len));
pos += oversize_used; //pos记录数据移动字节
oversize -= oversize_used; //更新oversize
space -= oversize_used;
}
Here we need to distinguish between space and oversize variables. A message has a size limit, and multiple pbufs can be hung under it. Space indicates how much space is left for the message; oversize indicates the size of the remaining memory in a pbuf. Therefore, the relationship between space and oversize is uncertain.
2. Create a new pbuf
After 1, if there is still remaining data that has not been added to unsent, and the pbuf can be added to the last segment, a new pbuf is created.
There is also a special case here, that is, the data sent is exactly continuous with the last pbuf in the memory. In this special case, no memory copy is needed, and only the size of the pbuf needs to be modified.
/*
* 将一个新的pbuf连接到pcb->unsent尾部
* 这里分复制内存和引用内存两种情况：
* 复制内存即：复制数据到新的内存空间
* 引用即：申请新的内存空间，并把指针指向数据的内存地址
*/
//最后一个pbuf已经满了且剩余有数据未复制，并且当前的报文还能再放pbuf的话，就在当前报文再添加一个pbuf
if ((pos < len) && (space > 0) && (last_unsent->len > 0)) {
u16_t seglen = LWIP_MIN(space, len - pos); //计算新的pbuf大小，要么是当前报文剩余的大小，要么是剩下数据的大小
seg = last_unsent;
//使用内存复制
if (apiflags & TCP_WRITE_FLAG_COPY) {
/* Data is copied */
//申请内存大小为seglen的pbuf
if ((concat_p = tcp_pbuf_prealloc(PBUF_RAW, seglen, space, &oversize, pcb, apiflags, 1)) == NULL) {
LWIP_DEBUGF(TCP_OUTPUT_DEBUG | LWIP_DBG_LEVEL_SERIOUS,
("tcp_write : could not allocate memory for pbuf copy size %"U16_F"\n",
seglen));
goto memerr;
}
//复制数据到新pbuf
TCP_DATA_COPY2(concat_p->payload, (const u8_t*)arg + pos, seglen, &concat_chksum, &concat_chksum_swapped);
queuelen += pbuf_clen(concat_p); //计算concat_p中pbuf的长度并添加到报文
} else {
//使用地址引用的方式
struct pbuf *p;
for (p = last_unsent->p; p->next != NULL; p = p->next); //找到last_unsent的最后一个pbuf
//!如果该pbuf是PBUF_ROM类型且内存与数据的内存是连续的，则直接扩展该pbuf的长度，不用新建
if (p->type == PBUF_ROM && (const u8_t *)p->payload + p->len == (const u8_t *)arg) {
LWIP_ASSERT("tcp_write: ROM pbufs cannot be oversized", pos == 0);
extendlen = seglen; //记下扩展长度
} else {
//其他类型的pbuf
//申请一块PBUF_ROM类型的pbuf，不给payload分配内存
if ((concat_p = pbuf_alloc(PBUF_RAW, seglen, PBUF_ROM)) == NULL) {
LWIP_DEBUGF(TCP_OUTPUT_DEBUG | LWIP_DBG_LEVEL_SERIOUS,
("tcp_write: could not allocate memory for zero-copy pbuf\n"));
goto memerr;
}
//将新的pbuf的payload指向数据地址，省去了复制的步骤
((struct pbuf_rom*)concat_p)->payload = (const u8_t*)arg + pos;
queuelen += pbuf_clen(concat_p); //计算concat_p中pbuf的长度并添加到报文
}
}
pos += seglen; //更新pos
}
} else {
//最后的pbuf足够放下数据
}
3. New message
If there is still data that has not been put into unsent after steps 1, 2, then a new message is created in the loop to store the data, and the message segment is inserted into the end of the unsent team. (There is no real enqueue in this step, but a local queue is created)
//循环创建报文，将剩余数据放入报文，报文入队
//pos已入队的数据，len数据总长度
while (pos < len) {
struct pbuf *p;
u16_t left = len - pos; //剩余数据长度
u16_t max_len = mss_local - optlen; //一个报文的最大长度
u16_t seglen = LWIP_MIN(left, max_len); //当前报文的大小，要么是最大的报文长度，要么是数据剩余长度
//使用内存复制
if (apiflags & TCP_WRITE_FLAG_COPY) {
//新建一个pbuf，大小是报文大小加上tcp首部
if ((p = tcp_pbuf_prealloc(PBUF_TRANSPORT, seglen + optlen, mss_local, &oversize, pcb, apiflags, queue == NULL)) == NULL) {
LWIP_DEBUGF(TCP_OUTPUT_DEBUG | LWIP_DBG_LEVEL_SERIOUS, ("tcp_write : could not allocate memory for pbuf copy size %"U16_F"\n", seglen));
goto memerr;
}
LWIP_ASSERT("tcp_write: check that first pbuf can hold the complete seglen",
(p->len >= seglen));
//复制数据到新pbuf
TCP_DATA_COPY2((char *)p->payload + optlen, (const u8_t*)arg + pos, seglen, &chksum, &chksum_swapped);
} else {
//使用引用：p2，p分别是tcp内容和tcp首部的pbuf，p2的payload指向数据地址
struct pbuf *p2;
//分配p2，PBUF_ROM类型不会给payload分配空间
if ((p2 = pbuf_alloc(PBUF_TRANSPORT, seglen, PBUF_ROM)) == NULL) {
LWIP_DEBUGF(TCP_OUTPUT_DEBUG | LWIP_DBG_LEVEL_SERIOUS, ("tcp_write: could not allocate memory for zero-copy pbuf\n"));
goto memerr;
}
//p2的payload指向数据区
((struct pbuf_rom*)p2)->payload = (const u8_t*)arg + pos;
//给报文头部分配p，如果分配失败，先释放p2
if ((p = pbuf_alloc(PBUF_TRANSPORT, optlen, PBUF_RAM)) == NULL) {
pbuf_free(p2);
LWIP_DEBUGF(TCP_OUTPUT_DEBUG | LWIP_DBG_LEVEL_SERIOUS, ("tcp_write: could not allocate memory for header pbuf\n"));
goto memerr;
}
//将首部和报文内容通过链表连接
pbuf_cat(p/*header*/, p2/*data*/);
}
queuelen += pbuf_clen(p); //更新报文队列长度
//如果发送队列或者缓存超出限制，则释放内存
if ((queuelen > TCP_SND_QUEUELEN) || (queuelen > TCP_SNDQUEUELEN_OVERFLOW)) {
LWIP_DEBUGF(TCP_OUTPUT_DEBUG | LWIP_DBG_LEVEL_SERIOUS, ("tcp_write: queue too long %"U16_F" (%d)\n",
queuelen, (int)TCP_SND_QUEUELEN));
pbuf_free(p);
goto memerr;
}
//给新的pbuf创建一个tcp_seg报文
if ((seg = tcp_create_segment(pcb, p, 0, pcb->snd_lbb + pos, optflags)) == NULL) {
goto memerr;
}
//如果是队列第一个报文段，直接引用
if (queue == NULL) {
queue = seg;
} else {
//将新报文连接到链尾
LWIP_ASSERT("prev_seg != NULL", prev_seg != NULL);
prev_seg->next = seg;
}
prev_seg = seg; //更新最后一个报文
LWIP_DEBUGF(TCP_OUTPUT_DEBUG | LWIP_DBG_TRACE, ("tcp_write: queueing %"U32_F":%"U32_F"\n",
lwip_ntohl(seg->tcphdr->seqno),
lwip_ntohl(seg->tcphdr->seqno) + TCP_TCPLEN(seg)));
pos += seglen; //移动pos
}
4. Put the data into the team
Careful friends found that the previous code did not actually add data to the unsent queue. In 1 we recorded oversize_used to indicate the length of the data filled in the last pbuf. In 2, we created a new pbuf. , In 3, we created a message queue.
In the following code, the data of the above three situations are entered into the unsent team one by one.
Fill in the pbuf data of the first stage
//如果last_unsent的最后一个pbuf有数据需要填入
if (oversize_used > 0) {
struct pbuf *p;
//找到unsent最后一个pbuf
for (p = last_unsent->p; p; p = p->next) {
p->tot_len += oversize_used; //所有pbuf的tot_len（理解）都要加上oversize_used
if (p->next == NULL) {
//在最后一个pbuf复制oversize_used大小的数据
TCP_DATA_COPY((char *)p->payload + p->len, arg, oversize_used, last_unsent);
p->len += oversize_used;
}
}
last_unsent->len += oversize_used; //报文长度也增加
}
pcb->unsent_oversize = oversize; //更新报文oversize
Connect the pbuf of the second stage to the message
//把新增了concat_p连接到该报文的pbuf链表
if (concat_p != NULL) {
LWIP_ASSERT("tcp_write: cannot concatenate when pcb->unsent is empty",
(last_unsent != NULL));
pbuf_cat(last_unsent->p, concat_p);
last_unsent->len += concat_p->tot_len; //更新链接后的报文长度
} else if (extendlen > 0) {
//内存扩展的情况： 内存连接在一起，直接把这个pbuf扩展了
struct pbuf *p;
LWIP_ASSERT("tcp_write: extension of reference requires reference",
last_unsent != NULL && last_unsent->p != NULL);
//所有pbuf的totlen都增加
for (p = last_unsent->p; p->next != NULL; p = p->next) {
p->tot_len += extendlen;
}
//由于数据地址与pbuf内存是连续的，所以不需要复制内存
p->tot_len += extendlen;
p->len += extendlen;
last_unsent->len += extendlen; //报文长度增加
}
Insert the local message queue into unsent, and finally update the sending window of tcp
if (last_unsent == NULL) {
pcb->unsent = queue;
} else {
last_unsent->next = queue;
}
pcb->snd_lbb += len;
pcb->snd_buf -= len; //发送buffer减少
pcb->snd_queuelen = queuelen; //更新发送队列长度
Three, summary
When you see this, you will understand that the tcp_write() function only inserts data into the unsent queue, and does not actually send the data out, and the function that actually sends the data out is tcp_output();
