LwIP Application Notes (3): Running LwIP Protocol Stack in RTOS Environment

foreword

This article is a follow-up to LwIP Application Notes (2): RAW API Migration Without Operating System Support . All the following content is based on the premise that RAW API porting has been completed. This article may not be too entangled in the code details, because the goal of this article is not to demonstrate how to write each line of code during the transplantation process, but to give the main process of transplantation on the basis of explaining the general framework, that is, in the transplantation process , what do we need to do.

1. Advantages and disadvantages of running in RTOS environment

In the non-RTOS environment, the user program interacts with the LwIP protocol stack through the callback interface, that is, the RAW API. The user tells the protocol stack what to do when certain events occur by registering a callback function. It can be understood that the user embeds part of the code written by the user into the code execution process of the protocol stack. The problem is that the callback function registered by the user will affect the operating efficiency of the protocol stack. Imagine such a scenario. In a non-RTOS environment, the user writes several different network function modules, which interact with the protocol stack through callbacks. Now suppose the user registers a callback function that needs to be executed for a long time. Once the protocol stack executes this callback function, it needs to wait for a long time for the callback to complete before processing other affairs. The network function module that registered this callback function successfully slowed down the entire network protocol by itself. The execution efficiency of the stack and other network function modules, no matter how efficient other network function modules are written, they must be subject to the slowest callback function. All that happens is as shown in the picture below.

At the same time, we can also foresee that as more and more callback functions are registered by the network function module, the protocol stack needs to spend more time to process the callback functions one by one.
The way to solve the above problems is to use RTOS to avoid the problem of user code slowing down the execution of the protocol stack by placing the LwIP protocol stack and user code in different threads for execution. Data transmission is performed between the user code and the protocol stack through the mailbox. In the RTOS environment, LwIP provides NETCONN API and Socket-like API for users to use, and user threads interact with protocol stack threads through these two sets of APIs. In this way, for the protocol stack, it no longer has to care about when to execute the user code, it only needs to process the data sent by the user code, and then throw the data that needs to be processed by the user code to the corresponding user processing thread. If unfortunately there is a user thread with low execution efficiency, it may lose the data that should be processed due to its own inefficiency, but it will not drag down the operating efficiency of other user threads and the protocol stack itself. The relationship between the user code and the protocol stack at this time is as follows.

However, we should also note that in RTOS, the user program and the LwiP protocol stack need to use means such as mailboxes to interact between threads, which will lead to additional system overhead and a certain loss of efficiency. In the case of extremely high performance requirements and relatively simple requirements, the RAW API interface based on callback functions may be more suitable.

2. RTOS related configuration items

It is necessary to modify the user configuration file lwipopts.h and add some configuration items related to the operating system, as shown in the figure below.

You need to focus on the configuration items under the operating system-related comments. The following configuration items need special attention:

• LWIP_COMPAT_MUTEX and LWIP_CMPAT_MUTEX_ALLOWED , after these two configurations are enabled, it is not necessary to transplant the mutex-related interface when transplanting the operating system abstraction layer. At this time, LwIP will use the binary semaphore to replace the function of the mutex lock, but this will not There is a way to deal with the problem of thread priority inversion, so you need to carefully consider whether to enable it. This function may be to take care of some operating systems that do not provide a mutex interface.
• xxx_RECVMBOX_SIZE and DEFAULT_ACCEPTMBOX_SIZE , such configuration items are used to configure the mailbox size used by each connection. The mailbox receives only the pointer address pointing to the actual data, so the size of each item is the pointer size. The values ​​of these items will be transmitted to the mailbox creation API of the operating system abstraction layer when creating a mailbox. If these values ​​are not configured, the mailbox size parameter passed in can also be ignored and a fixed value can be used when porting the mailbox creation API.

3. Migration of operating system abstraction layer

Since LwIP does not know what operating system it will run on, in order to improve portability, LwIP provides an operating system abstraction layer for docking with the function interface of the operating system. In general, the functional interfaces that the abstraction layer needs to implement are divided into the following categories:

• Semaphore-like function interface , which is used to provide semaphore-related operation interface;
• Mutex function interface , this type of function interface is used to provide the operation interface related to the mutex lock, if you set LWIP_COMPAT_MUTEX and LWIP_CMPAT_MUTEX_ALLOWED to 1 in the previous configuration, then this type of function interface does not need to be implemented;
• Mailbox function interface , this type of function interface is used to provide mailbox-related operation interfaces, if the operating system does not provide mailbox functions, you can consider using queues to implement;
• Thread management class function interface , which is used to provide operation interface related to thread creation;
• Other function-like interfaces , including initializing the operating system abstraction layer and obtaining the current system tick count time;

The figure below shows the various function interfaces that need to be transplanted during the transplantation process. For the specific comments of the functions and all the function interfaces provided by the abstraction layer of the operating system, you can refer to the sys.h file in LwIP. The comments in it detail the functions of each interface and Notes on porting, you need to refer to this file frequently during the porting process. At the same time, another valuable reference material is the official document generated based on doxygen. There is a section dedicated to the relevant content of the operating system abstraction layer. The official link is as follows: http://www.nongnu.org/lwip/2_1_x/group__sys__os .html . The author transplants the operating system abstraction layer based on rtthread nano. You can refer to the transplant files sys_arch.c and sys_arch.h

in the author's open source warehouse.

4. LwIP execution mode under the operating system

First of all, let’s recall how we run the LwIP protocol stack in the bare metal environment. After the protocol stack is initialized, we check whether the network card has received the message over and over again in the main loop. If the message is received, the message will be sent to protocol stack for processing. At the same time, we also need to periodically execute the sys_check_timeouts function responsible for handling timeout events. The execution process in the entire bare metal environment is as follows:

In the operating system environment, since LwIP runs alone in a thread (the thread name is determined by the TCPIP_THREAD_NAME macro definition, here is "tcpip_thread"), the only way to send a message to LwIP is through the mailbox. For this reason, LwIP implements tcpip_input function. Different from the netif_input function that directly throws the received message into the protocol stack for processing, the tcpip_input function will transfer the received message to the LwIP message receiving mailbox and wait for the LwIP thread to read and process it. In the initialization process, the netif_add function is called to add When the packet is received by the interface, use this function instead of the ethernet_input function to pass in.

If you trace the function calls inside LwIP, you can find that when the underlying link layer is Ethernet using the ARP protocol, the final method of processing packets is by calling the ethernet_input function. The difference is that in the bare metal environment, we can understand that the Ethernet message reading and message processing are in the same thread, and the function call is netif_input->ethernet_input; while under the real-time operating system, it is responsible for the Ethernet message The calling process of the thread reading the text is: tcpip_input->tcpip_inpkt->sys_mbox_trypost. At this point, the message is finally sent to the tcpip_mbox mailbox inside LwIP (the length of this mailbox is determined by the TCPIP_MBOX_SIZE macro definition above), and the LwIP thread will be in tcpip_mbox When there are pending packets in the mailbox, call ethernet_input to process these packets.

In this way, we can create a thread dedicated to reading messages from the network card (call this thread "eth_recv"), and then call the eth0_input function to read the message from the network card when receiving the message, and finally Send to the receiving mailbox of the LwIP thread through the tcpip_input function. When writing the eth_recv thread, do we have to keep polling whether there is a message like a bare metal machine? This is a bit of a waste of resources. At the same time, in order to ensure the timely receipt of messages, we will definitely assign a relatively high priority to this thread. If the non-stop polling keeps occupying CPU resources, other low-priority threads will not be able to execute. A better way is to use the interrupt hardware of the network card chip to trigger an external interrupt when a message is received, and notify the eth_recv thread of the arrival of a message through a semaphore during the interrupt. At this time, the eth_recv thread starts to read the message from the network card and Send it to the thread where LwIP is located. After all this is done, the eth_recv line hangs itself and waits for the next semaphore to arrive. In this way, excessive occupation of CPU resources by the polling method is avoided. The pile of words above is actually what the picture below expresses.

For specific and complete code implementation, please refer to https://gitee.com/water_zhang/enc28j60_arduino_shield_board/blob/master/software/stm32f412g_discovery_board_lwip/Middlewares/lwip/lwip_user/lwip_user_init.c

The following describes several key functions in the file:

• static void Lwip_User_IRQ_Callback(void *param)
  This function corresponds to the network card interruption in the above figure, and will be called when the network card interruption occurs. The only thing to do is to release the semaphore to notify the network card data packet receiving thread. The network card has something to send, and it should work up;
• void Lwip_User_Process(void *param)
  This function is the execution function of the network card data packet receiving thread eth_recv mentioned above. This thread will block the semaphore acquisition function until it receives the semaphore released by the interrupt. After receiving the semaphore, it will first Determine whether the interrupt is caused by receiving a data packet (the enc28j60 interrupt used by the author is a multi-source interrupt, you need to query the register to know the reason for triggering the interrupt), if it is a data packet receiving interrupt, call the Lwip_User_PollEthernetPacket function, this function will read the network card All data packets in and finally submit these data packets to the tcpip_mbox mailbox of the LwIP thread for processing;
• int Lwip_User_Init(void)
  This function implements the initialization of the entire protocol stack, which will be explained in the following initialization process section;

5. Initialization process

The initialization process in the bare-metal environment can be changed slightly. Here we will compare the two codes and give the differences. There is nothing to say. In order to facilitate the comparison of differences, some related statements that help debugging are deleted here, and only valuable content is kept.

6. Subsequent optimization points

In fact, we can know from the full text that we have not modified the network sending part, that is to say, at this time, the sending part of the underlying network card is still executed in the same thread as the LwIP thread. Later, we can consider making the sending part independent as a separate Thread, LwIP sends the data packets that need to be sent to the sending thread through the mailbox, the sending thread is blocked on the mailbox, and sends when there are data packets that need to be sent. This part of the modification needs to involve this file: https://gitee.com/water_zhang/enc28j60_arduino_shield_board/blob/master/software/stm32f412g_discovery_board_lwip/Middlewares/lwip/port/eth/eth0.c