fbus supports remote process calling methods similar to rpc, which is implemented using the invoke method. The invoke method provides multiple overloaded versions with different parameters, but ultimately it calls submit to submit the message or CBasejob object to the queue of the event loop.
Submit is a method provided by CFdbMessage, which specifically implements synchronous calls and asynchronous calls.
The following lists the relevant statements of invoke under the CFdbBaseObject class:
bool invoke(FdbSessionId_t receiver
, FdbMsgCode_t code
, IFdbMsgBuilder &data
, int32_t timeout = 0
, EFdbQOS qos = FDB_QOS_DEFAULT);
bool invoke(FdbSessionId_t receiver
, CFdbMessage *msg
, IFdbMsgBuilder &data
, int32_t timeout = 0);
bool invoke(FdbMsgCode_t code
, IFdbMsgBuilder &data
, int32_t timeout = 0
, EFdbQOS qos = FDB_QOS_DEFAULT);
bool invoke(CFdbMessage *msg
, IFdbMsgBuilder &data
, int32_t timeout = 0);
bool invoke(FdbSessionId_t receiver
, CBaseJob::Ptr &msg_ref
, IFdbMsgBuilder &data
, int32_t timeout = 0);
bool invoke(CBaseJob::Ptr &msg_ref
, IFdbMsgBuilder &data
, int32_t timeout = 0);
bool invoke(FdbSessionId_t receiver
, FdbMsgCode_t code
, const void *buffer = 0
, int32_t size = 0
, int32_t timeout = 0
, EFdbQOS qos = FDB_QOS_DEFAULT
, const char *log_info = 0);
bool invoke(FdbSessionId_t receiver
, CFdbMessage *msg
, const void *buffer = 0
, int32_t size = 0
, int32_t timeout = 0);
bool invoke(FdbMsgCode_t code
, const void *buffer = 0
, int32_t size = 0
, int32_t timeout = 0
, EFdbQOS qos = FDB_QOS_DEFAULT
, const char *log_data = 0);
bool invoke(CFdbMessage *msg
, const void *buffer = 0
, int32_t size = 0
, int32_t timeout = 0);
bool invoke(FdbSessionId_t receiver
, CBaseJob::Ptr &msg_ref
, const void *buffer = 0
, int32_t size = 0
, int32_t timeout = 0);
bool invoke(CBaseJob::Ptr &msg_ref
, const void *buffer = 0
, int32_t size = 0
, int32_t timeout = 0);
bool invoke(FdbMsgCode_t code
, IFdbMsgBuilder &data
, tInvokeCallbackFn callback
, CBaseWorker *worker = 0
, int32_t timeout = 0
, EFdbQOS qos = FDB_QOS_DEFAULT);
bool invoke(FdbMsgCode_t code
, tInvokeCallbackFn callback
, const void *buffer = 0
, int32_t size = 0
, CBaseWorker *worker = 0
, int32_t timeout = 0
, EFdbQOS qos = FDB_QOS_DEFAULT
, const char *log_info = 0);There are many invoke methods above, but they are functionally divided into two categories: one is asynchronous and the other is synchronous . To know whether it is synchronous or asynchronous, just look at the header file. A variety of invoke interfaces are listed above. These methods are all called by endpoints and are the entry points for user calls.
Above, CFdbBaseObject provides many invoke methods, and CFdbMessage also provides three invoke methods. The relationship between them, these invoke methods provided by CFdbBaseObject, no matter how transformed, the three invoke methods provided by CFdbMessage are finally called. The three invoke methods provided by CFdbMessage are declared as follows:
bool invoke(CBaseJob::Ptr &msg_ref
, uint32_t tx_flag
, int32_t timeout);
bool invoke(int32_t timeout = 0);
static bool invoke(CBaseJob::Ptr &msg_ref, int32_t timeout = 0);The previous article gave a detailed summary of the relationship between invoke. From the above we can conclude that there are two methods of calling invoke:
- Network endpoints, whether client or server, can call the invoke method message
- By defining a CFdbMessage instance, you can also directly call related methods, as follows
CFdbMessage msg;
msg->invoke(...);Okay, let’s get to the point. When I looked at the fdbus source code, I had this question. How is invoke synchronization implemented? What should I do if I want to achieve synchronization? With this question in mind, I took a closer look at the fdbus source code, and finally I understood how synchronization is implemented in the fdbus framework. I will try my best to present it below.
First understand the meaning of synchronous calls and asynchronous calls:
- Synchronous call: When calling a function, the calling function needs to wait for the called function to complete execution and return the result before executing the statement below the calling function. Before the execution logic of the called function is completed, it will not return, causing the calling function to block and wait. To take an extreme example, if a function takes 2s to complete, then the calling function will wait for 2s during this period. Execute the statements following it. To give the simplest example, all function calls we make in the same thread are synchronous calls. It should be added here that under normal circumstances, synchronous calls will include a blocking time. This time is to prevent blocking all the time. Of course, you can also choose to block all the time. Whether to set this blocking time is decided by the user according to his or her business needs.
- Asynchronous call: When calling a function, when the calling function calls a function to request a certain result, it is assumed that the called result takes 2s to execute. In the case of an asynchronous call, the called function will return immediately, allowing the calling function to continue. Execute the statements following it. Then after the execution is completed 2s later, the result is returned to the caller in some way. Of course, asynchronous calls are generally related to queues, event loops, and multi-threads. That is, in the case of multi-threads or multi-processes, the calling thread pushes the request into the event queue, and then the event loop traverses the event queue in sequence, and pushes it to the called thread or process for processing according to the type of event. When the called thread or process receives After receiving the event data, the result is returned to the calling thread or process in some way. Such a process is collectively called an asynchronous call.
This article only analyzes the synchronous calls of fdbus.
The following uses msg->invoke() as an example to introduce.
bool CFdbMessage::invoke(CBaseJob::Ptr &msg_ref , int32_t timeout)
{
auto msg = castToMessage<CFdbMessage *>(msg_ref);
return msg ? msg->invoke(msg_ref, FDB_MSG_TX_SYNC, timeout) : false;
}From the above code, you can see that the synchronous calling interface of invoke under CFdbMessage contains two parameters, one is the shared pointer pointing to the message object, and the other is the timeout. You can see that another invoke method in CFdbMessage is called in the above code. I wonder if you have noticed the second parameter. Yes, it is the FDB_MSG_TX_SYNC parameter. From the literal meaning, everyone understands that this is setting the synchronization flag. Source code As follows:
bool CFdbMessage::invoke(CBaseJob::Ptr &msg_ref
, uint32_t tx_flag
, int32_t timeout)
{
mType = FDB_MT_REQUEST;
return submit(msg_ref, tx_flag, timeout);
}In the above code, tx_flag is a message flag. All the invoke methods mentioned before ultimately require calling this invoke method for logical processing.
bool CFdbMessage::submit(CBaseJob::Ptr &msg_ref
, uint32_t tx_flag
, int32_t timeout)
{
....
bool sync = !!(tx_flag & FDB_MSG_TX_SYNC);
if (sync && mContext->isSelf())
{
setStatusMsg(FDB_ST_UNABLE_TO_SEND, "Cannot send sychronously from context");
return false;
}
if (tx_flag & FDB_MSG_TX_NO_REPLY)
{
mFlag |= MSG_FLAG_NOREPLY_EXPECTED;
}
else
{
mFlag &= ~MSG_FLAG_NOREPLY_EXPECTED;
if (sync)
{
mFlag |= MSG_FLAG_SYNC_REPLY;
}
if (timeout > 0)
{
mTimer = new CMessageTimer(timeout);
}
}
bool ret = true;
if (tx_flag & FDB_MSG_TX_NO_QUEUE)
{
dispatchMsg(msg_ref);
}
else
{
setCallable(std::bind(&CFdbMessage::dispatchMsg, this, _1));
if (sync)
{
ret = mContext->sendSync(msg_ref);
}
else
{
ret = mContext->sendAsync(msg_ref);
}
}
...
}bool CBaseWorker::sendSync(CBaseJob::Ptr &job, int32_t milliseconds, bool urgent)
{
return send(job, milliseconds, urgent);
}bool CBaseWorker::send(CBaseJob::Ptr &job, int32_t milliseconds, bool urgent)
{
if (job->mSyncReq)
{
return false;
}
// now we can assure the job is not in any worker queue
CBaseJob::CSyncRequest sync_req(job.use_count());
job->mSyncReq = &sync_req;
if (!send(job, urgent))
{
return false;
}
if (job->mSyncReq)
{
job->mSyncLock.lock();
if (job->mSyncReq)
{
if (milliseconds <= 0)
{ // job->mSyncLock will be released
sync_req.mWakeupSignal.wait(job->mSyncLock);
}
else
{ // job->mSyncLock will be released
std::cv_status status = sync_req.mWakeupSignal.wait_for(job->mSyncLock,
std::chrono::milliseconds(milliseconds));
if (status == std::cv_status::timeout)
{ // timeout! nothing to do.
}
}
job->mSyncReq = 0;
}
job->mSyncLock.unlock();
}
return true;
}The condition variable mWakeupSignal is blocked through the wait method.
bool CBaseWorker::send(CBaseJob::Ptr &job, bool urgent, bool swap)
{
bool ret;
if (mExitCode || !mEventLoop)
{
return false;
}
if (urgent)
{
ret = mUrgentJobQueue.enqueue(job, swap);
}
else
{
ret = mNormalJobQueue.enqueue(job, swap);
}
return ret;
}Although it is a synchronous call, the send method is called before the condition variable mWakeupSignal blocks. From the send method source code, you can see that although it is a synchronous call, the message data or message object is still pushed into the event queue, and then proceeds sequentially through the event loop. Processing, the difference is that the wait method is called through the condition variable mWakeupSignal to block the calling thread, and then waits to receive the reply message and wakes up the calling thread.
The above code is the logic of the sending link. The key source code for unblocking the calling thread is listed below:
void CBaseSession::doResponse(CFdbMessageHeader &head)
{
bool found;
PendingMsgTable_t::EntryContainer_t::iterator it;
CBaseJob::Ptr &msg_ref = mPendingMsgTable.retrieveEntry(head.serial_number(), it, found);
if (found)
{
auto msg = castToMessage<CFdbMessage *>(msg_ref);
auto object_id = head.object_id();
if (msg->objectId() != object_id)
{
LOG_E("CFdbSession: object id of response %d does not match that in request: %d\n",
object_id, msg->objectId());
terminateMessage(msg_ref, FDB_ST_OBJECT_NOT_FOUND, "Object ID does not match.");
mPendingMsgTable.deleteEntry(it);
delete[] mPayloadBuffer;
mPayloadBuffer = 0;
return;
}
auto object = mContainer->owner()->getObject(msg, false);
if (object)
{
msg->update(head, mMsgPrefix);
msg->decodeDebugInfo(head);
msg->replaceBuffer(mPayloadBuffer, head.payload_size(), mMsgPrefix.mHeadLength);
mPayloadBuffer = 0;
auto type = msg->type();
doStatistics(type, head.flag(), mStatistics.mRx);
if (!msg->sync())
{
switch (type)
{
case FDB_MT_REQUEST:
object->doReply(msg_ref);
break;
case FDB_MT_SIDEBAND_REQUEST:
object->onSidebandReply(msg_ref);
break;
case FDB_MT_GET_EVENT:
object->doReturnEvent(msg_ref);
break;
case FDB_MT_SET_EVENT:
default:
if (head.type() == FDB_MT_STATUS)
{
object->doStatus(msg_ref);
}
else
{
LOG_E("CFdbSession: request type %d doesn't match response type %d!\n",
type, head.type());
}
break;
}
}
}
msg_ref->terminate(msg_ref);
mPendingMsgTable.deleteEntry(it);
}
}The entry handler function that receives the response.
void CBaseJob::terminate(Ptr &ref)
{
if (!ref)
{
return;
}
if (mSyncReq)
{
mSyncLock.lock();
if (mSyncReq)
{
// Why +1? because the job must be referred locally.
// Warning: ref count of the job can not be changed
// during the sync waiting!!!
if (ref.use_count() == (mSyncReq->mInitSharedCnt + 1))
{
mSyncReq->mWakeupSignal.notify_one();
mSyncReq = 0;
}
}
mSyncLock.unlock();
}
}
You see, the condition variable here is awakened through notify_one.
The above code should be able to clearly see the implementation of fdbus synchronous call. Actually just 3 steps:
- The calling thread pushes the message into the message queue
- After pushing into the message queue, block the calling thread through the condition variable and wait for wake-up.
- After receiving the reply message, send notify_one to wake up the blocked thread
Here is a little more verbose that I think is more important. We all know that functions have input parameters, output parameters, and input and output parameters. That is, the parameters in a function can be both input parameters and output parameters. Then a synchronous call occurs in fdbus. How do the parameters CBaseJob::Ptr &msg_ref compile the content of the received message?
Please see the article below
Summarize
By analyzing fdbus synchronization implementation, I learned about a synchronization implementation method and gained a deeper understanding of synchronous calls. Of course, fdbus is based on the network, so even synchronous calls essentially need to be pushed into the message queue or event queue. According to processed asynchronously.