A brief introduction to Thrift

foreword

I watched Thrift again recently and recorded it

text

1. What is Thrift

Origin of Thrift

Thrift is an interface description language and binary communication protocol used to define and create cross-language services. It is used as a remote procedure call (RPC) framework and was developed by Facebook for "large-scale cross-language service development". It combines a software stack through a code generation engine to create varying degrees of seamless cross-platform efficient services, using C#, C++ (based on POSIX compliant systems) Cappuccino, Cocoa, Delphi, Erlang, Go, Haskell, Java , Node.js, OCaml, Perl, PHP, Python, Ruby and Smalltalk programming languages. Open sourced by Facebook in 2007, entered the Apache incubator in May 2008, and became a top-level project of Apache in October 2010.

what is RPC

Remote Procedure Call (RPC) is a computer communication protocol. The protocol allows a program running on one computer to call a subroutine on another computer without the programmer having to additionally program this interaction. Such as Java RMI (Remote Method Invocation (Remote
Method Invocation). It enables an object on a Java virtual machine to call a method on an object in another Java virtual machine as if calling a local object).

As can be seen from the above figure, RPC itself is a client-server model and a request-response protocol.
Some implementations extend the model of remote invocation and implement two-way service invocation, but in any case, the invocation process is still initiated by a client, and the server provides the response, and the basic model remains unchanged.
The calling process of the service is as follows:
1. The client calls the client stub, which is a local procedure call
. 2. The client stub packages the parameters into a message, and then sends the message. The packaging process is also called marshalling
3. The system where the client is located sends the message to the server
4. The system of the server passes the received package to the server stub
5. The server stub unpacks to get the parameters. Unpacking is also known as unmarshalling
6. Finally, the server stub calls the service process. The returned result is passed to the client in the opposite steps

Other RPC frameworks

The current RPC framework generally has two different focus directions, one focuses on service governance, and the other focuses on cross-language calls.
1. Service governance RPC frameworks include Dubbo, Motan, etc. Such RPC frameworks are characterized by rich functions, providing high-performance remote calls, service discovery and governance functions, and suitable for microservice splitting and management of large-scale services. , for a specific language (Java) project can be very friendly and transparent access. But the disadvantage is that the language coupling is high, and cross-language support is difficult.
2. Cross-language calling RPC frameworks include Thrift, gRPC, etc. This type of RPC framework focuses on cross-language calling of services and can support language-independent calls in most languages, which is very suitable for providing general-purpose services for different languages. Remote service scenarios. However, this type of framework does not have a mechanism for service discovery. In actual use, the proxy layer generally needs to perform request forwarding and load balancing policy control.

RPC VS RESTFUL

The message transmission of RPC can be through TCP, UDP or HTTP, etc. RPC is similar to RESTful when it comes to transferring messages over HTTP, but there are differences.
1. In terms of usage, the client and server sides of RPC are tightly coupled. RESTful operates resources based on http semantics. The order of parameters generally does not matter, and it is easy to convert links and resource locations through proxies. From this point of view, RESTful is more flexible. And they operate on different objects. RPC operates on method objects. RESTful operations operate on resources, not methods.
2. From a performance point of view, when using Http, Http itself provides a wealth of status functions and extension functions, but it is also because Http provides too many functions that it needs to carry more information during network transmission. From a performance point of view On the contrary, it is relatively inefficient. On the other hand, the RPC service network only transmits data related to the business content, and the transmitted data is smaller and has higher performance.

Thrift's protocol stack structure

Thrift is a C/S architecture system. The top layer is the business logic code implemented by the user. The second layer is the code automatically generated by the Thrift compiler, which is mainly used for parsing, sending and receiving structured data. The main task of TServer is to efficiently accept client requests and forward requests to Processor for processing. Processor is responsible for responding to client requests, including RPC request forwarding, call parameter parsing, user logic calls, and return value writing. The following parts from TProtocol are thirft's transport protocol and underlying I/O communication. TProtocol is used for data type analysis, which converts structured data into byte stream to TTransport for transmission. TTransport is a transport layer closely related to the underlying data transmission, responsible for receiving and sending message bodies in the form of byte streams, regardless of the data type. The underlying IO is responsible for the actual data transmission, including sockets, files, and compressed data streams.

type of data

1. Base Types: base types
2. Struct: Structure type
3. Container: container type, namely List, Set, Map
4. Exception: exception type
5. Service: defines the interface of the object, and a series of methods

protocol

Thrift allows you to select the type of transmission communication protocol between the client and the server. The transmission protocol is generally divided into text and binary transmission protocols. In order to save bandwidth and provide transmission efficiency, it is generally used. The binary type of transmission protocol is the majority, but sometimes the text-based protocol is still used, which needs to be based on the actual needs in the project/product:

1. TBinaryProtocol – Binary encoding format for data transfer.
2. TCompactProtocol - This protocol is very efficient and uses Variable-Length Quantity (VLQ) encoding to compress data.
3. TJSONProtocol - Data encoding protocol using JSON for data transfer.
4. TSimpleJSONProtocol - This thrift provides JSON-only write-only protocol, suitable for parsing via scripting languages.
5. TDebugProtocol – It is used to help developers debug during the development process, and it is displayed in the form of text for easy reading.

transport layer

1. TSocket - uses blocking I/O for transmission and is the most common mode.
2. TFramedTransport - uses a non-blocking way to transmit by block size, similar to NIO in Java.
3. TFileTransport- As the name implies, it is transmitted in the way of files. Although this method does not provide Java implementation, it is very simple to implement.
4. TMemoryTransport - uses memory I/O, just like the ByteArrayOutputStream implementation in Java.
5. TZlibTransport - Use to perform zlib compression, does not provide a Java implementation.

server type

1. TSimpleServer - Single-threaded server side using standard blocking I/O.
2. TThreadPoolServer - Multithreaded server side using standard blocking I/O.
3. TNonblockingServer - Multithreaded server side uses non-blocking I/O and implements NIO channels in Java.

2. Why use Thrift

application

1. Facebook's open source log collection system (scribe: https://github.com/facebook/scribe )
2. Taobao's real-time data transmission platform (TimeTunnel http://code.taobao.org/p/TimeTunnel/wiki/index )
3. Evernote Open Interface ( https://github.com/evernote/evernote-thrift )
4. Quora(http://www.quora.com/Apache-Thrift)
5. HBase( http://abloz.com/hbase/book.html#thrift )

Advantages and disadvantages

1. A very large number of language bindings are supported.
2. The thrift file generates object code, which is easy to use.
3. Separation of data structure and transport representation, support for multiple message formats.
4. Contains a complete client/server stack for fast RPC implementation.
5. Both synchronous and asynchronous communication are supported.
6. Like protobuf (Google's flexible and efficient language platform-independent method for representing structured data), it does not support dynamic features

3. How to use Thrift

Simple demo of Thrift

First configure the Thrift environment, write a hello.thrift, the content is as follows

namespace java service.demo
service Hello{
    string helloWorld(1:string para)
}

Execute the following command to generate java code

thrift -r -gen java Hello.thrift

Generate the gen-java folder, copy the Hello.java inside to the
pom.xml file in the project and add dependencies

<dependencies>
<dependency>
    <groupId>org.apache.thrift</groupId>
    <artifactId>libthrift</artifactId>
    <version>0.10.0</version>
</dependency>
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-log4j12</artifactId>
    <version>1.7.5</version>
</dependency>
</dependencies>

Implementation class

package service.demo;

import org.apache.thrift.TException;

/**
 * 1.服务实现类
 *
 * @author micheal
 * @create 2018-03-26 11:03
 **/
public class HelloServiceImpl implements Hello.Iface{
    @Override
    public String helloWorld(String para) throws TException {
        return para+":helloWorld";
    }
}

service class

package service.demo;

import com.sun.org.apache.xpath.internal.SourceTree;
import org.apache.thrift.TProcessor;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.server.TServer;
import org.apache.thrift.server.TSimpleServer;
import org.apache.thrift.transport.TServerSocket;
import org.apache.thrift.transport.TTransportException;

/**
 * 2.服务端
 *
 * @author micheal
 * @create 2018-03-26 11:04
 **/
public class HelloServiceServer {
    public static void main(String[] args) throws TTransportException {
        System.out.println("服务端开启......");
        TProcessor tProcessor=new Hello.Processor<Hello.Iface>(new HelloServiceImpl());
        TServerSocket serverSocket = new TServerSocket(8999);
        TServer.Args tArgs = new TServer.Args(serverSocket);
        tArgs.processor(tProcessor);
        tArgs.protocolFactory(new TBinaryProtocol.Factory());
        TServer server = new TSimpleServer(tArgs);
        server.serve();
    }
}

client

package service.demo;

import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;

/**
 * 3.客户端
 *
 * @author micheal
 * @create 2018-03-26 11:44
 **/
public class HelloServiceClient {
    public static void main(String[] args) throws TException {
        System.out.println("客户端启动.....");
        TTransport transport =new TSocket("localhost", 8999, 30000);
        // 协议要和服务端一致
        TProtocol protocol = new TBinaryProtocol(transport);
        Hello.Client client = new Hello.Client(protocol);
        transport.open();
        String result = client.helloWorld("TOM");
        System.out.println(result);
    }
}

Start the server first
,

客户端启动.....
Received 1
TOM:helloWorld

Summarize

This brief introduction will give you a general understanding of the principle of the RPC framework and the protocol stack of Thrift, which will be introduced in detail below.