Analysis of postmaster code in Postgres (middle)

Today we discuss the following details of the postmaster:

backend的启动和client的连接请求的认证
客户端取消查询时的处理
接受pg_ctl的shutdown请求进行shutdown处理

2. Interaction with the front end

2.1 The initiation of backend and the authentication of the client's connection request

Regarding the startup of backend, its function call stack is as follows:

PostmasterMain()
　  |->ServerLoop()
　      |->initMasks()
　      |->for(;;)
　          |->select()         <--监听端口
　          |->ConnCreate()     <--创建connection相关的数据结构
　          |->BackendStartup() <--建立后端进程backend process
　              |->PostmasterRandom()
　              |->canAcceptConnections()
　              |->fork_process()
　              |->InitPostmasterChild()
　              |->ClosePostmasterPorts()
　              |->BackendInitialize()
　                  |->ProcessStartupPacket()
　              |->BackendRun()
　                  |->PostgresMain()
            |->ConnFree()       <--释放connection相关的数据结构

Simply put, in the system call select(), we listen to the client's connection request. When we read a client request, we will create a related data structure for it and do some initialization. Note that at this time, it is only listening to accept the request, and whether the request is legal (for example, whether the password is correct) is not judged at this time. The judgment is placed in BackendStartup().

You may be wondering: BackendStartup() is used to create the backend, but is it a bit late to verify after the backend is created?

Let's look at the processing of BackendStartup() (for details, see the call stack and source above).

typedef struct bkend
{
    pid_t       pid;            /* process id of backend */
    long        cancel_key;     /* cancel key for cancels for this backend */
    int         child_slot;     /* PMChildSlot for this backend, if any */

    /*
     * Flavor of backend or auxiliary process.  Note that BACKEND_TYPE_WALSND
     * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
     * bkend_type is normal, you should check for a recent transition.
     */
    int         bkend_type;
    bool        dead_end;       /* is it going to send an error and quit? */
    bool        bgworker_notify;    /* gets bgworker start/stop notifications */
    dlist_node  elem;           /* list link in BackendList */
} Backend;

(The data structure of backend is more important, I will quote it here first)

First, the program calls the PostmasterRandom() function to generate a cancelkey. What does this cancelkey ​​do? It is used to mark the cancel command sent by the front-end: that is, when the front-end sends an instruction to abort the operation of the current SQL text (for example, you press Crtl+C), the postmaster will first find the corresponding command in the backendList of the backend through the backendPID first. backend, and then use this cancelkey ​​to verify with this backend (this will be mentioned in the next section).

Then call canAcceptConnections to determine whether the current postmaster status can accept connections? The reader is puzzled again: hasn’t the connection been accepted before? The previous select just called the select() system call to obtain the connection request (not even a connection request, but only received a packet from the client, which may be startup_packet or cancel_request_packet). As for whether it is acceptable or not Connection, we have two judgments:

• Is the current database in a state that can accept connections? (startup/shutdown/inconsistent recovery state cannot accept connections);
• Is the current number of data connections full (more than MaxConnections)

If the conditions are not met. We mark the backend to be started as dead_end. That is to say, this backend is only used to report errors to the frontend, and exit immediately after an error is reported. So we don't assign Slot to it. After judging that it can be connected, we assign a slot to it.

Keep going down. Call fork_process() to start a process (of course it is used as a backend). After the backend is started, we can leave the postmaster and leave everything behind to be handled by the backend itself. The accompanying InitPostmasterChild() is used to initialize the backend process and switch the environment handle from postmaster to backend. Then call ClosePostmasterPorts() to close the file descriptors that are not needed at this time.

Then call BackendInitialize for further initialization. What we are more interested in here may be that it calls ProcessStartupPacket() to get the StartupPacket sent by the front end and allocate memory for it, and do some simple judgment processing. The verification part is placed at the back.

Finally, we call BackendRun() to actually run the backend.
From the code, we can see that the BackendRun() function is just a shell, it just switches the memory context to TopMemoryContext and obtains some parameters specified by the -o parameter on the postmaster command line, and then passes these parameters to the PostgresMain() function to Here, we see that the PostgresMain() function is very similar to PostmasterMain(). All are command entries. And later we see that there is also a Loop in the PostgresMain() function. It is to read the SQL text sent by the client.

The InitPostgres() function is a very important initialization function called by PostgresMain(). Naturally, its role is to initialize. What initialization to do?

I list some:

InitProcessPhase2()         Add my PGPROC struct to the ProcArray
SharedInvalBackendInit()    shared cache invalidation communication(inval)
ProcSignalInit()            Register the current process in the procsignal array
RegisterTimeout()           Register timeout
RelationCacheInitialize()   Initialize relationcache
InitCatalogCache()          Initialize catalog cache
InitPlanCache()             Initialize callbacks for inval

EnablePortalManager()       Portals are objects representing the execution state of a query,
                            This module provides memory management services for portals

InitializeClientEncoding()  initialize client encoding

I won't say much about PostgresMain(), which is very similar to PostmasterMain(), except that all the objects processed are for banckend, see the code for details.

Here we can answer why we need to create a backend before doing the verification:

The postmaster only acts as an intermediary, but involves too much shared memory and other operations that can cause errors, making the postmaster main program more robust and stable. At the same time, if I spend time doing verification in ServerLoop, I think it is too time-consuming.

Oh, forgot, we haven't talked about the authentication of the client's connection request. Here is the function call stack:

BackendRun()
    |->PostgresMain()
        |->InitPostgres()
            |->PerformAuthentication()
                |->ClientAuthentication()

PerformAuthentication() is called after EnablePortalManager() in InitPostgres. At this time, most of the initialization work of the backend process itself has been completed.

The entry for Client authentication here is ClientAuthentication(). Its main work is as follows:

hba_getauthmethod() //获取hba文件中和该条请求匹配的auth method
|
v
switch(auth_method)//根据auth_method和请求信息做出相应的处理
|
v
status == STATUS_OK ? sendAuthRequest() : auth_failed() 
                    //根据返回值决定向client发送验证packet还是拒绝请求

It should be noted that when the verification fails and the client's request is rejected, the program will report an error and exit here. In this way, this backend is the backend of a dead_end, which will exit under the command of the postmaster. For details, see the following sections.

---

2.2 Intermediary when the client cancels the query

When we run a very long SQL query in the client (such as the psql command line) (not to say that the query must be very long, but if the time is too short, you will not have time to cancel~), at this time for various reasons you Want to abort the query, so you press Crtl+C. Immediately displayed on the client:

postgres=# select * from test order by id asc ;
Cancel request sent
ERROR: canceling statement due to user request

Let's take a look at how postgres handles such cancels.

First picture:

Corresponding to the above figure, we list the function call stacks for the involved processes:
client:
The psql command calls setup_cancel_handler() during initialization to register a signal handler function before the MainLoop of psql. After pressing Ctrl+C), call handle_sigint() to handle this signal. After successful processing, print:

Cancel request sent

(src/bin/psql/startup.c)
main()
    |->setup_cancel_handler()
        |->pqsignal(SIGINT, handle_sigint)
    |->successResult = MainLoop(stdin)

postmaster:processCancelRequest: After receiving the packet sent by the client, the postmaster establishes a backend process (backend) to process it. When it finds that it is a cancel_request_packet, it calls the processCancelRequest() function to process the packet, and sends the corresponding backend through the PID. Send a SIGINT signal.

PostmasterMain()
　  |->ServerLoop()
　      |->for(;;)
　          |->BackendStartup() <--建立后端进程backend process
　              |->BackendInitialize()
　                  |->ProcessStartupPacket()
　                      |->processCancelRequest()
　                          |->signal_child(bp->pid, SIGINT)

backend:pqsignal(SIGINT, StatementCancelHandler): The following signal processing function is registered on the postgresMain() function, which accepts the SIGINT signal sent by the postmaster, performs corresponding processing, and sets two global variables:

pqsignal(SIGINT, StatementCancelHandler)
{
...
        InterruptPending = true;
        QueryCancelPending = true;
...
}

And these two global variables determine whether CHECK_FOR_INTERRUPTS() takes effect:

#define CHECK_FOR_INTERRUPTS() \
do { \
    if (InterruptPending) \
        ProcessInterrupts(); \
} while(0)

We enter the ProcessInterrupts() function and find that it is used to process the client's abort request:

ProcessInterrupts(){
...
InterruptPending = false;
...
        {
            LockErrorCleanup();
            ereport(ERROR,
                    (errcode(ERRCODE_QUERY_CANCELED),
                     errmsg("canceling statement due to user request")));
        }
...     
}

The error message is the one we saw above.

---

2.3 Accept the shutdown request of pg_ctl

We often use pg_ctl to control the postgres server, such as start, stop and reload, etc. The start parameter corresponds to the startup of the server, which we have discussed in Postgres postmaster code analysis (above) . Here we discuss the processing of specifying the stop parameter.

Before we start the discussion, let's take a look at the enumeration type PMState.

typedef enum
{
    PM_INIT,                    /* postmaster starting */
    PM_STARTUP,                 /* waiting for startup subprocess */
    PM_RECOVERY,                /* in archive recovery mode */
    PM_HOT_STANDBY,             /* in hot standby mode */
    PM_RUN,                     /* normal "database is alive" state */
    PM_WAIT_BACKUP,             /* waiting for online backup mode to end */
    PM_WAIT_READONLY,           /* waiting for read only backends to exit */
    PM_WAIT_BACKENDS,           /* waiting for live backends to exit */
    PM_SHUTDOWN,                /* waiting for checkpointer to do shutdown
                                 * ckpt */
    PM_SHUTDOWN_2,              /* waiting for archiver and walsenders to
                                 * finish */
    PM_WAIT_DEAD_END,           /* waiting for dead_end children to exit */
    PM_NO_CHILDREN              /* all important children have exited */
} PMState;

This enumeration type marks the current state of the database. where PM_RUN is a watershed. From PM_INIT to PM_RUN, the database gradually transitions from the initialization state to the normal running state. From PM_RUN to PM_NO_CHILDREN, the database gradually transitions from the normal running state to the state that can be closed. Understanding this helps us understand the timing of database startup and shutdown. The comments after each state above can already explain the transition conditions between each state well, so I won't go into details here.

For the stop parameter of pg_ctl, we have three modes:

model	sent singal	Processing of singal
smart	SIGTERM	Wait for children to end their work, then shut down
fast	SIGINT	Abort all children with SIGTERM (rollback active transactions and exit) and shut down when they are gone
immediate	SO MUCH	abort all children with SIGQUIT, wait for them to exit, terminate remaining ones with SIGKILL, then exit without attempt to properly shut down the database system.

Here we will discuss the smart mode first, and other modes are actually similar.

First execute "pg_ctl stop -m smart", this time is actually sending a SIGTERM signal to the postmaster;

The postmaster receives the SIGTERM signal, triggers pqsignal(SIGTERM, pmdie), and calls the pmdie() function to process the SIGTERM signal;

pmdie
    |->SignalSomeChildren(SIGTERM,BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
                                        向autovacuum和bgworker子进程转发SIGTERM信号
    |->PostmasterStateMachine()         更新数据库的状态PM_State

The processing in pmdie is shown above. pmdie calls SignalSomeChildren() to send a SIGTERM signal to the specified process, and these processes themselves also have signal processing functions, which are processed and terminated after receiving the SIGTERM signal of the postmaster. (After the child process terminates, a SIGCHLD signal is sent to the parent process, which is an inherent processing of the operating system). PostmasterStateMachine() is a tool function, which is called in many signal processing functions of postmaster to update PM_State according to the current PM_State of the database and the life and death of related processes.

This time we look at the backend process:

backend:pqsignal(SIGTERM, die);   //die

The signal processing function of the backend process itself calls the die function process to exit after receiving the SIGTERM signal.

The topic goes back to the postmaster. When it receives the SIGCHLD signal of the child process, it triggers pqsignal(SIGCHLD, reaper), and the reaper() function is called to process the SIGCHLD signal sent by the child process:

reaper()
    |->switch(PID)      根据PID类型判断子进程类型，分别进行处理
    |->PostmasterStateMachine()     更新数据库的状态PM_State

In this way, the postmaster will always receive the SIGCHLD signal of the child process, and update PM_State after corresponding processing.

So when is it sure that all child processes have ended? Or look at the PostmasterStateMachine() function:

PostmasterStateMachine() 
            当最后一个backend(dead_end)结束时，reaper处理子进程通过调用PostmasterStateMachine更新当前状态，
            将当前状态由PM_WAIT_DEAD_END转换为PM_NO_CHILDREN时：
            PM_WAIT_DEAD_END  -> pmState = PM_NO_CHILDREN
            说明说有子进程都已退出，postmaster调用ExitPostmaster结束自身：
            ExitPostmaster()

---

---

This is the discussion in this section, and the next time is ready to discuss:

后端process的管理
DB的shoutdown的处理
backend异常结束时的处理
BootstrapMain()的处理

Set up the flag first, lest you forget it. Welcome everyone to like it~