Pipeline based on multi-process concurrency-process communication

1. Pipeline

The so-called pipeline is a series of buffers (Pipe) in the kernel. The data written by a process from one end of the pipeline is actually cached in the kernel, and the other end reads, that is, reads this piece of data from the kernel.

characteristic:

• There are two types of pipes: anonymous pipes and named pipes (also called named pipes)
• Simple implementation
• size limit
• unformatted byte stream data
• Data copy between user mode and kernel mode
• Anonymous pipe: one-way communication, not cross-PC
• Famous pipeline: two-way communication, cross-PC
• Built-in synchronization and mutual exclusion
• A string of buffers (Pipe) in the kernel

2. Anonymous channels

1 Introduction

Anonymous pipes can only be used for communication between parent and child processes, not cross-network communication, and communication is one-way. Also, piped data is an unformatted stream with a limited size.

Under normal circumstances, the input and output of the console process are in the console window, but if we specify its input and output when creating a child process, the child process will read data from our pipeline and write the output data to The pipeline we specify.
This is why standard input is redirected in our code, otherwise system("pause") would have no effect.

2. Parent-child process: communication process of anonymous pipe?

The parent process writes to the pipe, and the child process reads the pipe. The process is as follows.

• 1. The parent process, CreatePipe() will get two handles when creating a pipe, that is, the read handle of the pipe and the write handle of the pipe.
• 2. When the parent process executes CreateProcess() to start the child process,You can pass the handle (such as passing the read handle to the child process) to the child process, the write handle is saved by the parent process itself.
• 3. The parent process calls WriteFile() to write data to the pipeline.
• 4. The child process calls GetStdHandle() to obtain the read handle of the pipeline,
• 5. Then, the child process ReadFile() reads data from the pipeline.

In this way, each of the two processes has a handle (parent process write handle, child process read handle), and the two processes can write and read the same pipe file through their respective handles to achieve cross-process communication.

• The above process says that the parent process writes the pipe, and the child process reads the pipe. Of course, it can also be reversed, the child process writes and the parent process reads, that is, the parent process gives the write handle to the child process, and the parent process saves the read handle).

3. Related functions

3.1. Create pipeline CreatePipe

BOOL CreatePipe(
PHANDLE hReadPipe; //指向管道读句柄
PHANDLE hWritePipe; //指向管道写句柄
LPSECURITY_ATTRIBUTES lpPipeAttributes;  //指向管道安全属性
DWORD nSize; //管道大小
)

3.2, write pipeline WriteFile

BOOL WriteFile(
               HANDLE  hFile,//文件句柄
               LPCVOID lpBuffer,//数据缓存区指针
               DWORD   nNumberOfBytesToWrite,//要写的字节数
               LPDWORD lpNumberOfBytesWritten,//用于保存实际写入字节数的存储区域的指针
               LPOVERLAPPED lpOverlapped//OVERLAPPED结构体指针
);

3.3, read pipeline ReadFile

BOOL ReadFile(
    HANDLE hFile, //文件的句柄
    LPVOID lpBuffer, //接收数据的缓冲区
    DWORD nNumberOfBytesToRead,    //读取的字节数
    LPDWORD lpNumberOfBytesRead,    //指向实际读取字节数的指针
    LPOVERLAPPED lpOverlapped
    //如文件打开时指定了FILE_FLAG_OVERLAPPED，那么必须，用这个参数引用一个特殊的结构。
    //该结构定义了一次异步读取操作。否则，应将这个参数设为NULL
);

3.4. Get the handle GetStdHandle
This function is used to get the handle of the specified standard device (standard input, standard output or standard error). This demo is used for: the child process obtains the read handle passed by the parent process.

HANDLE WINAPI GetStdHandle( _In_ DWORD nStdHandle);

parameter	meaning
STD_INPUT_HANDLE	standard input handle
STD_OUTPUT_HANDLE	standard output handle
STD_ERROR_HANDLE	standard error handler

4、demo

• parent process

//main.cpp
#include <windows.h> 
#include "iostream"
#define BUFSIZE 4096 
using namespace std;
 
int main(int argc, char* argv[])
{
    
    
    cout << "\n ** This is a message from the father process. ** \n";

    cout << "第一步：创建管道" << endl;

    // Set the bInheritHandle flag so pipe handles are inherited. 
    SECURITY_ATTRIBUTES saAttr;
    saAttr.nLength = sizeof(SECURITY_ATTRIBUTES);
    saAttr.bInheritHandle = TRUE;
    saAttr.lpSecurityDescriptor = NULL;

    // Create a pipe for the child process's STDIN. 
    HANDLE handle_read;
    HANDLE handle_write;
    bool ret = CreatePipe(&handle_read, &handle_write, &saAttr, 0);
    if (!ret)
    {
    
    
        cout << "创建进程失败：create pipe fail" << endl;
    }

    //设置写句柄不可以被子进程继承,不设置也不影响。 Ensure the write handle to the pipe for STDIN is not inherited. 
    if (!SetHandleInformation(handle_write, HANDLE_FLAG_INHERIT, 0))
    {
    
    
        cout << "设置句柄失败：set handle fail!" << endl;
    }

    cout << "第一步：子进程、设置管道句柄的继承" << endl; 
    {
    
    
        // Create the child process. 

        char cmdline[] = "childprocess.exe";
        PROCESS_INFORMATION piProcInfo;
        // Set up members of the PROCESS_INFORMATION structure. 
        ZeroMemory(&piProcInfo, sizeof(PROCESS_INFORMATION));

        // Set up members of the STARTUPINFO structure. // This structure specifies the STDIN handles for redirection.
        STARTUPINFO si;
        ZeroMemory(&si, sizeof(STARTUPINFO));
        si.cb = sizeof(STARTUPINFO);
        si.hStdInput = handle_read; //把管道的读句柄传给子进程
        si.dwFlags |= STARTF_USESTDHANDLES;

        // Create the child process.
        ret = CreateProcess(NULL, cmdline, NULL, NULL, TRUE, CREATE_NEW_CONSOLE, NULL, NULL, &si, &piProcInfo);
        if (!ret)
            cout << "创建子进程失败：create child process faile!";
        else
        {
    
    
            cout << "创建子进程成功：create child process sucess!";
            // Close handles to the child process and its primary thread.
            CloseHandle(piProcInfo.hProcess);
            CloseHandle(piProcInfo.hThread);

            CloseHandle(handle_read);
        }
    }

    cout << "第三步：向管道中写入数据：write to pipe." << endl;
    {
    
    
        // write contents to the pipe for the child's STDIN.

        DWORD len;
        char chBuf[BUFSIZE] = " hello pipe";
        for (int i = 0; i < 10; i++)
        {
    
    
            bool ret = WriteFile(handle_write, chBuf, sizeof(chBuf), &len, NULL);//子进程读了后，父进程才可以继续写入管道
            if (!ret)
            {
    
    
                cout << "写入管道数据失败。i=" << i << endl;
                break;
            }
            else 
            {
    
    
                cout << "send data " << i << " is:" << chBuf << endl;
            }
            
        }
        cout << "写入管道数据结束。" << endl;
        // Close the pipe handle so the child process stops reading. 
        if (!CloseHandle(handle_write))
            cout << "colse handle fail" << endl;
    }
    return 0;
}

• child process

// main.cpp
#include <windows.h>
#include"iostream"
#define BUFSIZE 4096 
using namespace std;
 
int main(int argc, char* argv[])
{
    
     
    cout<<"\n ** This is a message from the child process. ** \n";
   CHAR chBuf[BUFSIZE]; 
   DWORD len; 
   HANDLE handle_read; 
 
   handle_read = GetStdHandle(STD_INPUT_HANDLE);
   if (handle_read == INVALID_HANDLE_VALUE)
      ExitProcess(1); 
 
   for (int i = 0;i<5;i++)
   {
    
     
   // Read from standard input and stop on error or no data.
      bool ret = ReadFile(handle_read, chBuf, BUFSIZE, &len, NULL);
      if (!ret || len == 0)
      {
    
    
          cout << "读取数据失败" << endl;
          break;
      }

      cout << "receive data "<<i<<" is:" << chBuf << endl;
   } 
   cout << "读取数据结束" << endl;
   Sleep(5000);
   freopen("CON", "r", stdin);    // 重定向输入,否则system("pause")会无效
   //CloseHandle(handle_read);
   system("pause");
   return 0;
}

5. Results

• parent process
  
• child process
  
• Postscript
  1. After the child process takes out the data in the pipeline, the parent process can continue to write data into the pipeline.
  2. After the parent process enters the fifth send data, the child process does not read it. But when the child's sleep is over, for some reason, it is not sure who read the fifth input.
  3. The output before the sleep function of the child process is as follows
  

3. Famous pipelines

1 Introduction

Named pipe, as the name suggests, this pipe must have a name. Use the name of the pipe to ensure that multiple processes access the same pipe. In fact, named pipes can not only transfer data between different processes on the same computer, but also support reliable, one-way or two-way data communication between different processes on different computers across a network.

The difference between the server and client of a named pipe is that
the server is the only process that has the right to create a named pipe, and only he can accept the connection request of the pipe client.
The client can only establish a connection with the named pipe server of one thread.

• Named pipes have good flexibility in use, as shown in:
  　　1) It can be used both locally and on the network.
  　　2) Can be referenced by its name.
  　　3) Support multi-client connection.
  　　4) Support two-way communication.
  　　5) Support asynchronous overlapping I/O operations

2. Between two processes: the working process of named pipes

• Server
  Named pipe server, the steps are as follows:
  1) Use the API function CreateNamedPipe to create a named pipe instance handle.
  2) Use the API function ConnectNamedPipe to monitor client connection requests on the named pipe instance.
  3) Use the two API functions ReadFile and WriteFile to receive data from the client or send data to the client.
  4) Use the API function DisconnectNamedPipe to close the named pipe connection.
  5) Use the API function CloseHandle to close the named pipe instance handle.
• Client
  Named pipe client, the steps are as follows:
  1) Call the API function WaitNamedPipe to wait for a named pipe instance to be available.
  2) Call the API function CreateFile to open a named pipe instance and establish a connection.
  3) Call the API functions WriteFile and ReadFile to send data to and receive data from the server respectively.
  4) Call the API function CloseHandle to close the opened named pipe session.

3. Related functions

3.1. Create a named pipe CreateNamedPipe
creates an instance of a named pipe and returns a handle for subsequent pipe operations.

HANDLE CreateNamedPipeA(
  [in]           LPCSTR                lpName,
  [in]           DWORD                 dwOpenMode,
  [in]           DWORD                 dwPipeMode,
  [in]           DWORD                 nMaxInstances,
  [in]           DWORD                 nOutBufferSize,
  [in]           DWORD                 nInBufferSize,
  [in]           DWORD                 nDefaultTimeOut,
  [in, optional] LPSECURITY_ATTRIBUTES lpSecurityAttributes
);

The first parameter lpName in the CreateNamedPipe function interface: .\pipe\pipename must be in this format. The "." in the middle means the local machine. If you want to establish a connection with the remote machine, you need to set the name of the remote server.

3.2. Listening request ConnectNamedPipe
listens to the client connection request on the named pipe instance.

BOOL ConnectNamedPipe(
  [in]                HANDLE       hNamedPipe,
  [in, out, optional] LPOVERLAPPED lpOverlapped
);

3.3. Wait for a named pipe instance WaitNamedPipe
to wait until the timeout interval expires or the instance of the specified named pipe is available for connection

BOOL ConnectNamedPipe(
  [in]                HANDLE       hNamedPipe,
  [in, out, optional] LPOVERLAPPED lpOverlapped
);

4、demo

• Server

#include <iostream>
#include <windows.h>
using namespace std;

int main()
{
    
    
	printf("创建命名管道并等待连接\n");

	char pipeName[] = "\\\\.\\Pipe\\mypipe";
	HANDLE hPipe = CreateNamedPipe(pipeName, PIPE_ACCESS_DUPLEX, PIPE_TYPE_MESSAGE | PIPE_READMODE_MESSAGE | PIPE_WAIT
		, PIPE_UNLIMITED_INSTANCES, 0, 0, NMPWAIT_WAIT_FOREVER, 0);


	//waiting to be connected
	if (ConnectNamedPipe(hPipe, NULL) != NULL)
	{
    
    
		printf("连接成功，开始发送数据\n");

		DWORD    dwWrite;
		const char* pStr = "data from server";
		if (!WriteFile(hPipe, pStr, strlen(pStr), &dwWrite, NULL))
		{
    
    
			cout << "write failed..." << endl << endl;
			return 0;
		}
		cout << "sent data: " << endl << pStr << endl << endl;
	}

	DisconnectNamedPipe(hPipe);
	CloseHandle(hPipe);//关闭管道
	printf("关闭管道\n");
	system("pause");
}

• client

// ClientPip.cpp 

#include <iostream>
#include <windows.h>
using namespace std;
#define BUFSIZE 5


int main()
{
    
    

	printf("命名管道:客户端上线\n");
	printf("按任意键以开始连接命名管道\n");
	getchar();
	printf("开始等待命名管道\n");

	char pipeName[] = "\\\\.\\Pipe\\mypipe";
	if (WaitNamedPipe(pipeName, NMPWAIT_WAIT_FOREVER) == FALSE)
		return 0;

	printf("打开命名管道\n");
	HANDLE hPipe = CreateFile(pipeName, GENERIC_READ | GENERIC_WRITE, 0,
		NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);

	if ((long)hPipe == -1)
		return 0;


	//接收服务端发回的数据
	BOOL fSuccess = false;
	DWORD len = 0;
	char buffer[BUFSIZE];
	string recvData = "";
	do
	{
    
    
		fSuccess = ReadFile(hPipe, buffer, BUFSIZE * sizeof(char), &len, NULL);
		char buffer2[BUFSIZE + 1] = {
    
     0 };
		memcpy(buffer2, buffer, len);
		recvData.append(buffer2);
		if (!fSuccess || len < BUFSIZE)
			break;
	} while (true);

	cout << "recv data:" << endl << recvData.c_str() << endl << endl;

	FlushFileBuffers(hPipe);
	DisconnectNamedPipe(hPipe);
	CloseHandle(hPipe);

	system("pause");
	return 0;
}

5. Output

If there are any mistakes or deficiencies, welcome to comment and point out! Creation is not easy, please indicate the source for reprinting. If it is helpful, remember to like and follow (⊙o⊙)
For more content, please follow my personal blog: https://blog.csdn.net/qq_43148810