Giraph source code analysis (two) - Start Master / Worker Services

Author | White Pine

Note: This article is original, reprint reference number to be associated with Lan.

1, org.apache.giraph.bsp.CentralizedService Interface

功能:Basic service interface shared by both CentralizedServiceMaster and CentralizedServiceWorker.

Giraph source code analysis (two) - Start Master / Worker Services

2, org.apache.giraph.bsp.CentralizedServiceMaster Interface

功能:At most, there will be one active master at a time, but many threads can be trying to be the active master.

3、org.apache.giraph.bsp.CentralizedServiceWorker接口

功能:All workers should have access to this centralized service to execute the following methods.

4, org.apache.giraph.bsp.BspService abstract class

功能:Zookeeper-based implementation of CentralizedService.

5、org.apache.giraph.master.BspServiceMaster类

功能:ZooKeeper-based implementation of CentralizedServiceMaster.

6、org.apache.giraph.worker.BspServiceWorker类

功能:ZooKeeper-based implementation of CentralizedServiceWorker.

BspServiceWorker class has WorkerClient and WorkerServer examples, were used as IPC client and server communication, transmits data by Netty. Examples of objects actually NettyWorkerClient WorkerClient, WorkerServert instance objects actually NettyWorkerServer.

NettyWorkerClient implements WorkerClient接口,NettyWorkerServer implements WorkerServer接口。

Create a NettyServer NettyWorkerServer the object constructor class, for the bottom of the IPC communications, there is a ServerData object as the data entity. ServerData contains partitionStore, edgeStore, incomingMessageStore, currentMessageStore, the aggregate value of the Worker, and the like.

Create a NettyClient NettyWorkerClient the object constructor class, for the bottom of the IPC communications, as a client.

7, org.apache.giraph.worker.InputSplitsCallable abstract classes, inheritance Callable interface.

Giraph source code analysis (two) - Start Master / Worker Services

Function : input vertex or edge for loading the splits, each thread has a WorkerClientRequestProcessor example (actually NettyWorkerClientRequestProcessor object), responsible for sending data to a remote worker.

Giraph source code analysis (two) - Start Master / Worker Services

WorkerClient objects NettyWorkerClientRequestProcessor for transmitting the object is inside WorkerClient BspServiceWorker objects.

readInputSplit VertexInputSplitsCallable class () method for reading information from the split vertex, the object then calls sendVertexRequest NettyWorkerClientRequestProcessor () method sends the vertex Partition it belongs to.

8, org.apache.giraph.graph.ComputeCallable classes, inheritance Callable interface.

Complete "Calculation - Communications - Synchronization" in the subject process. Each thread has a WorkerClientRequestProcessor example (actually NettyWorkerClientRequestProcessor object), responsible for sending data to a remote worker.

Guess you like

Origin blog.51cto.com/14463231/2422575