Sequoia kernel notes | Session (Session)

SequoiaDB Sequoia database is a financial level distributed relational database, adhere to start from scratch to build a distributed open source database engine. "Kernel Notes Series" designed to share design ideas and code analysis exchange SequoiaDB giant sequoias database engine, to help communities achieve user-depth understanding of the principles SequoiaDB and build a database of open source and open technology ecosystem.

01 Basic Concepts

Session connection is easy to confuse the two concepts. Session (Session) refers to a context (Context) during the end of the communicating parties from the start of communication to communicate. This context is a piece of memory on the server side, the client machine records this connection, which application and which user logs on and so on through the information. It means connected to one end to the physical path from the client database instance. The network connection can be established, through IPC mechanisms can be established in the machine. Usually to establish a connection between a dedicated server or a dispatcher with the client process.

Design of the session 02 SequoiaDB

Distributed Database SequoiaDB usually consists of a cluster data node, the coordinator node and catalog nodes, there are various types of connections between nodes within the cluster. Thus there are several sessions SequoiaDB, and the different sessions corresponding to different services. The main task for the session end request is sent to the communication process.

A typical session cluster structure shown below.

The coordinating node listens client initiates a connection request, the connection is established to create a coord session, that is, the coordinator node session, after they all requests by the corresponding process this session initiated by the client. coord session after receiving the client to process the request, will bind cataloging information for analysis, which confirmed the need to send the node to perform, catalog may be sent to the node, one or more data nodes. Data node upon receiving the message sent by the coordinator node, a shard session is created to handle the request, performs the specified operation, and returns the data.

03 communication plane

Between nodes in the cluster SequoiaDB plurality of communication channels, different types of nodes to provide different services. In order to ensure that each of these services to work properly, SequoiaDB provides a plurality of communication nodes plane. Briefly, one communication plane corresponding to a service port, different port provide different types of services, which is why the installation SequoiaDB, required in reserved port number within a range of.

SequoiaDB currently provides the following several communication plane:

local 平面(local service): 使用节点配置文件中指定的基础服务端口号 svcname

repl 平面(repl service): 使用端口号svcname+1

shard 平面(shard service): 使用端口号svcname+2

cat 平面(cat service): 使用端口号svcname+3

rest 平面(rest service): 使用端口号svcname+4

om 平面(om service): 使用端口号svcname+5

04 local session

Local session is created when directly connected nodes (ie configuration svcname). Direct relatively broad meaning, referring to an arbitrary node connected to the local service port. When a client connects to the coordinator node, it will create a local session on the coordinator node. When the monitor receives a new connection request on the local port, will create a new session (memory structures) and a service thread (execution unit), and bind (attach) them together. Follow the client will interact directly with this new service thread.
Code Guide

Various types of inheritance session SequoiaDB below in FIG.

It can be seen from the figure, the local session, the increment / total amount of the synchronization session, the session replication, are inherited from the same base class _ISession. The following will be combined with the networking session in which several key introduction, mainly session establishment / time to destroy the structure of the session, and operation.

Local session corresponding to the data structure is a class _pmdLocalSession, the main function of the thread is _pmdLocalSession :: run (), starts after the conversation thread, to circulate inside this function, message received and processed, the loop is exited until the session is about to end.

Local session can bind different processor to perform different processes. For the coordinator node, the binding is _pmdCoordProcessor. For cataloging nodes and data nodes, binding is _pmdDataProcessor. For the coordinator node will first call _pmdCoordProcessor interface for message processing, if it can not identify the type of request, it will be called again _pmdDataProcessor interface for processing.

05 sync (or copy) session Repl Session

分区组内的节点之间，通过同步动作来保证数据的一致性。同步分为两种，一种是正常运行状态下的增量同步，一种是异常情况下的全量同步。同步是通过对应的同步会话与同步线程来处理的，它涉及到两个节点，在数据生产方称为源端，在数据消费方称为目标端。由于只有数据节点和编目节点上会进行数据复制，所以只有在这两种类型的节点上，才存在同步会话。
1）增量同步会话

增量同步会话分为增量同步源端会话和目标端会话，且存在于复制组正常运行期间。在数据节点和编目节点的启动过程中，主节点或从节点都会开启增量同步的监听。同时，它也会主动启动一个增量复制目标端会话，并向它选定的源端发送同步请求。源端节点上会被动创建一个增量同步源端会话，这两个会话后续会开始进行交互，以完成数据同步。

2）全量同步会话

全量同步会话存在于全量同步期间，在集群正常运行期间及全量同步完成后不存在。与增量同步会话一样，全量同步会话也分为源端和目标端。

需要全量同步的场景有三种:

备节点的重放速度跟不上主节点，主节点上复制日志绕接，导致备节点还未获取到的复制日志被覆盖，备节点无法继续增量同步 

节点异常重启，启动后节点根据读取到的异常启动状态决定全量同步

节点正常停止后正常重启，但停止时间较长，期间其它节点上的日志已经发生了绕接

而无论是上述哪种情况，都会先发生增量复制会话。当这些原因导致增量同步无法继续进行的时候，目标节点上会主动创建一个全量同步会话(以及对应的线程)，并退出当前的增量复制线程。当全量同步会话启动时，会向源端发送一个全量同步开始的消息。此时源端上会被动创建一个全量同步源端会话。至此，全量同步的会话创建完成，后续这两个会话之间会开始进行交互，完成数据同步。
代码导读

四种会话对应的类为: _clsReplSrcSession,  _clsReplDstSession,  _clsF***cSession,  _clsFSDstSession。

同步相关的会话都是异步会话，上述四种会话使用同一个会话管理器：_clsReplSessionMgr 来进行管理。

异步会话响应的消息类型及对应的处理函数，一般在对应的类中通过 OBJ_MSG_MAP 等宏进行定义，请参考代码。 

06 会话的查看

用户可通过 db.snapshot ( SDB_SNAP_SESSIONS ) 命令列出当前数据库节点中的所有会话，或通过 db.snapshot ( SDB_SNAP_SESSIONS_CURRENT ) 命令列出当前数据库节点中的当前会话。
代码导读

session derived classes _monSessionFetcher operation implemented in the class, and () in the function data preparing init. Users can choose to view the current session (the current thread eduCB export interfaces) or all sessions (using _pmdEDUMgr interface to export). After preparing the data, called by the upper class unified framework of context fetch interface to obtain data.