Scrcpy Source Code Analysis Series
[Screencasting] Scrcpy Source Code Analysis 1 (Compilation)
[Screencasting] Scrcpy Source Code Analysis 2 (Client - Connection Stage)
[Screencasting] Scrcpy Source Code Analysis 3 (Client - Screencasting Stage)
[Screencasting] 】Srcpy source code analysis four (final chapter - Server chapter)

In the previous article, we analyzed the directory structure and compilation method of the Scrcpy project. At the beginning of this article, we will go deep into the source code.

As mentioned above, Scrcpy is a screen projection software, which is divided into Client (computer) and Server (mobile phone). In this article, we will analyze the client part first.

1. Introduction to the principle

Before analyzing the code, let's talk about screen projection. Screen projection, as the name implies, is to project the interface of device A to device B. Now the application fields of screen projection are getting wider and wider, and there are more and more tricks, such as TV projection, car projection, mobile phone projection and so on. According to the author's imprecise classification, there are three main types of presentations on the market that can be called "screencasting":

The first type: directly project the entire interface on A as it is, that is, the mirror image mode. This type of screencasting is usually a way to record screens and transmit video streams. For example, AirPlay's mirroring mode, MiraCast, Music Broadcasting, etc.;
The second category: push mode, the scene comparison scene of playing video. That is, A transmits a connection to B, and B plays by itself, and A can transmit some simple control commands after learning. Such as DLNA protocol, etc.;
The third category: projection of some applications or functions based on special protocols, mostly in the automotive field. For example, Apple's CarPlay, Huawei HiCar, Baidu CarLife, etc.

Our protagonist Scrcpy belongs to the first category. The principle is roughly to establish a connection between the mobile phone side and the computer side, and then record the screen on the mobile phone side, and continuously send the video stream to the computer side for decoding and rendering to the interface. In addition to mirroring the screen, Scrcpy also supports computer-side anti-control and file upload. Now let's follow the code to see the details.

2. Client logic

Because the overall process is long, let's look at it in two stages: connection and screen projection:

2.1 Connection phase

The main function of the program is main.cin, and the function is executed immediately at random main_scrcpy():

// main.c
int
main(int argc, char *argv[]) {
    
    
#ifndef _WIN32
    return main_scrcpy(argc, argv);
#else
	// ...参数字符串相关的处理...
    int ret = main_scrcpy(wargc, argv_utf8);
	return ret;
#endif
}

It can be seen that the main_scrcpy method will be executed regardless of whether it is a windows platform. Scrcpy is cross-platform and can run on Windows, Linux, and MacOS, but the main process in the code is basically the same. Most of the macro judgments involving the system platform are the differences between string processing, system data structures, and system library functions. Has no effect on our analysis.

// main.c
int
main_scrcpy(int argc, char *argv[]) {
    
    
	// ...
  	struct scrcpy_cli_args args = {
    
    
    	.opts = scrcpy_options_default
    };
    scrcpy_parse_args(&args, argc, argv)
    
	av_register_all();
	
	scrcpy(&args.opts);
	// ...
}

main_scrcpyWe only need to pay attention to three methods in the function :

scrcpy_parse_args(&args, argc, argv)- Parse the parameters passed in by the user to replace the default parameters.
av_register_all()- Scrcpy uses FFmpeg to decode the video stream from the mobile phone. This function is a library function of FFmpeg. When programming FFmpeg, this function is usually called at the beginning for FFmpeg initialization.
scrcpy(&args.opts)- scrcpy.cThe function to call, and pass in parameters.

The following are scrcpy()some key codes of the function ( scrcpy()the function as a whole is relatively long and can be divided into the connection phase and the screen projection phase, let’s look at the connection phase first):

// scrcpy.c
enum scrcpy_exit_code
scrcpy(struct scrcpy_options *options) {
    
    
	// [连接阶段]
	// ...
	// 初始化SDL事件子系统
	SDL_Init(SDL_INIT_EVENTS)
	
	// 声明参数
	struct sc_server_params params = {
    
    
		// 有很多参数，没有贴全，贴两个作为示例
		.max_fps = options->max_fps,
		.encoder_name = options->encoder_name,
	}

	// 声明连接状态回调
	static const struct sc_server_callbacks cbs = {
    
    
        .on_connection_failed = sc_server_on_connection_failed,
        .on_connected = sc_server_on_connected,
        .on_disconnected = sc_server_on_disconnected,
    };

	// 初始化，将参数和回调添加到相应结构体中
	sc_server_init(&s->server, &params, &cbs, NULL);

	// 启动Server端
	sc_server_start(&s->server);
	
	// 初始化SDL视频子系统
	SDL_Init(SDL_INIT_VIDEO);
	
	// 等待连接状态
	await_for_server(&connected);

	// [投屏阶段]
	// ...
}

In the connection phase, we need to pay attention to several parts:

SDL_Init(SDL_INIT_EVENTS)- SDL is an open source cross-platform multimedia development library. It can be used to develop window programs and provides a rich event system. The Scrcpy window is developed with SDL. Here is the SDL library function used to initialize the SDL event subsystem.
struct sc_server_params params- Declare parameters, which are used to store the parameters when starting the program in the structure.
struct sc_server_callbacks cbs- Declare the status callback function for the connection.
sc_server_init- Initialize some structures, including storing parameters and state callbacks in the structure.
sc_server_start- Start the server side.
SDL_Init(SDL_INIT_VIDEO)- Initialize the SDL video subsystem.
await_for_server- Waiting for connection status.

Among them, 5 and 7 that need to be focused on , we will focus on them.

2.1.1 `sc_server_start`- Start Server

To start the server side is to start the program on the mobile phone side. If we let ourselves implement the program on the mobile phone side from the computer, what can we think of? Yes, it is adb. Scrcpy also starts the Server side through adb, but there are still many details in it. Now let's chase the code.

In the function sc_server_start, a new thread is started to execute run_serverthe function.

// server.c
bool
sc_server_start(struct sc_server *server) {
    
    
    sc_thread_create(&server->thread, run_server, "scrcpy-server", server);
}

run_serverThe function is the core of starting the Server, and its key parts are as follows:

// server.c
static int
run_server(void *data) {
    
    
	// 执行adb start-server
	sc_adb_start_server(&server->intr, 0);
	// 执行adb devices -l
	sc_adb_select_device(&server->intr, &selector, 0, &device);
	// tcpip方式连接，执行adb connect连接
	if (params->tcpip) {
    
    
		sc_server_configure_tcpip_unknown_address(server, device.serial);
	}
	// 执行adb push
	push_server(&server->intr, serial);
	// 执行adb reverse 或 adb forward进行端口映射
	sc_adb_tunnel_open(&server->tunnel, &server->intr, serial,
                            params->port_range, params->force_adb_forward);
	// 执行app_process
	execute_server(server, params);
	// 创建一个进程观察者监听进程结束
	sc_process_observer_init(&observer, pid, &listener, server);
	// 进行业务连接
	sc_server_connect_to(server, &server->info);
	// 触发on_connected回调
	server->cbs->on_connected(server, server->cbs_userdata);
	// 等待条件变量cond_stopped发生改变
	while (!server->stopped) {
    
    
        sc_cond_wait(&server->cond_stopped, &server->mutex);
    }
	// 结束进程
	sc_process_terminate(pid);
}

run_serverFor this function, we need to pay attention to several parts:

sc_adb_start_server()- Chasing this function is actually calling the command adb start-serverto start the adb service.
sc_adb_select_device()- Similar to 1, the execution is adb devices -lto view the device list,
sc_server_configure_tcpip_unknown_address()- If it is in tcpip mode, the - parameter is brought when starting the scrcpy program -tcpip. This function will be executed, and after chasing it, you can see that two sub-functions are called:

3.1. sc_server_switch_to_tcpip（）- Switch to tcpip connection mode, here is a small logic:
1. First adb shell ip routeobtain the ip address of the target device;
2. Execute adb -s serial shell getprop service.adb.tcp.portget port number. If the port number exists, it means that the device has enabled the tcpip connection function, skip it.
3. If the port number is empty, perform adb tcpip 5555the function of enabling tcpip connection, and set the port number to 5555.
3.2. sc_server_connect_to_tcpip()- Execute adb connect ip:portthe adb connection.
push_server()- By adb -s serial pushuploading /usr/local/share/scrcpy/scrcpy-server(as mentioned in the compilation article, this directory is where ninja put server.apk in this directory during installation) to /data/local/tmp/scrcpy-server.jar(this is a fixed directory).
sc_adb_tunnel_open()- Execute adb reverse -s serial reverse localabstract:scrcpy tcp:<PORT>port mapping, here is to reverse proxy the Unix domain socket named scrcpy on the mobile phone to the PORT port on the PC (the default port number is 27183). Then create a socket with port number 27183. Let me explain here that at the business level, the mobile phone serves as a server to provide video streams and control commands. But at the implementation level of the network layer, the computer side acts as the server side, monitors the socket connection and waits for the mobile phone side to connect. The advantage of this is that the PC terminal can start the service before the mobile terminal, and the mobile terminal can directly connect. On the contrary, it may be necessary for the PC side to retry continuously when connecting, and the connection cannot be made until the mobile phone side starts and starts listening.

For adb port mapping, you can read my other article - Introduction to ADB port mapping and LocalServerSocket .
execute_server()- By app_processopening the server program, the complete command is adb -s serial shell CLASSPATH=/data/local/tmp/scrcpy-server.jar app_process / com.genymobile.scrcpy.Server 1.25 [PARAMS] . In this way, the Server class of the mobile phone side program can be executed. Because there are many methods that need to hook the system on the mobile phone side, app_processthe execution authority can be improved by using app_process to call the high-privilege API .
sc_process_observer_init()- Start a new thread to create an observer to monitor the end of the process, and it will always be blocked there, and the sc_process_observer_initcallback will be triggered after the process stops.
sc_server_connect_to()- Waiting for the connection on the mobile phone side. As mentioned earlier, the PC side is used as the server at the network level, so this will eventually call the accept4connection waiting for the mobile phone side. The mobile phone side will connect twice, and you can get two video_sockets and control_sockets respectively (here The socket encapsulates the platform structure of Linux and Windows, which is a unified data structure), and is used to transmit video streams and control instructions.
server->cbs->on_connected- Trigger the connection success callback. You should remember that this callback scrcpy()is registered as sc_server_on_connecteda function in the previous function. In this function, it will eventually be called SDL_PushEvent()and the event will be sent through the SDL event mechanism EVENT_SERVER_CONNECTED. This event will be sent to await_for_serverthe function in the next section, which pre-buried a foreshadowing, we have an impression.
sc_cond_wait(&server->cond_stopped)- Wait for the condition variable cond_stoppedto change, otherwise, if the program is not terminated, it will be blocked and no subsequent code will be executed. In this function, the built-in function of SDL is used to wait for the condition variable. SDL_CondWaitIt can be seen that if SDL is used to develop upper-layer applications, many ready-made functions provided by SDL itself can be used, such as the event mechanism above and the thread synchronization strategy here.
sc_process_terminate(pid)- end process. In the previous step, if the condition variable has not changed, it will be blocked until this step. Until cond_stoppeda change occurs, it will come to this step and end the process. The Unix platform finally calls the library function kill(_pid_t, int), and the Windows platform calls it TerminateProcess(HANDLE, UINT).

So far, sc_server_startthe main process of the function has basically been chased. To sum up, the function of this function is to upload the server-side program to the target device through adb, and then app_processstart the server-side program to initiate a connection with itself to obtain two sockets forvideo_socket subsequent use. control_socketFinally send an EVENT_SERVER_CONNECTEDevent. The overall process is basically as shown in the figure below, and the remaining problem is EVENT_SERVER_CONNECTEDwhere to send it. We fill in behind.

insert image description here

2.2.2 `await_for_server`- Waiting for connection state

Next, let's continue to chase functions. If you can't remember the calling order, it doesn't matter. Let's look at the more streamlined structure of await_for_serverthe caller function:scrcpy()

// scrcpy.c
enum scrcpy_exit_code
scrcpy(struct scrcpy_options *options) {
    
    
	// [连接阶段]
	// ...
	// 启动Server端（异步子线程）
	sc_server_start(&s->server);
	// 等待连接状态
	await_for_server(&connected);
	// ...
	
	// [投屏阶段]
}

scrcpy()After the function sc_server_start()creates a child thread to execute the operation, await_for_server()the function will be called immediately. Let's see await_for_server()what's done in:

// scrcpy.c
static bool
await_for_server(bool *connected) {
    
    
    SDL_Event event;
    while (SDL_WaitEvent(&event)) {
    
    
        switch (event.type) {
    
    
            case SDL_QUIT:
                LOGD("User requested to quit");
                *connected = false;
                return true;
            case EVENT_SERVER_CONNECTION_FAILED:
                LOGE("Server connection failed");
                return false;
            case EVENT_SERVER_CONNECTED:
                LOGD("Server connected");
                *connected = true;
                return true;
            default:
                break;
        }
    }

    LOGE("SDL_WaitEvent() error: %s", SDL_GetError());
    return false;
}

That's right, the above is await_for_server()the entire content of the function, that is to say, it uses the SDL event system and has been waiting for the connection result event. If you can't wait for the target event, you can't get out of this function.

Remember EVENT_SERVER_CONNECTEDthe incident mentioned in the previous paragraph? That's right, that's what's waiting here. After the connection is established between the mobile phone side and the PC side, sc_server_on_connectedthe callback will eventually send an event. After receiving the target event, you can jump out of the loop and go down. scrcpy()The function also ends the connection phase and enters the screen projection phase.

Then our flowchart can be filled completely.

insert image description here

2.2 Timing diagram

Here is a timing diagram of the client-side connection phase. Different colors in the diagram represent different threads.

scrcpy-client-connect

3. Summary

In this article, we explored the logic of the Scrcpy Client connection phase. The points involved include FFmpeg initialization, SDL event system and synchronization mechanism, ADB port mapping, app_process running Android programs, etc. In the next article, we will continue to explore the screen projection stage on the client side.

[Screen projection] Scrcpy source code analysis 2 (Client articles - connection stage)

Client article - connection phase

1. Introduction to the principle

2. Client logic

2.1 Connection phase

2.1.1 `sc_server_start`- Start Server

2.2.2 `await_for_server`- Waiting for connection state

2.2 Timing diagram

3. Summary

Guess you like

[Screen projection] Scrcpy source code analysis 2 (Client articles - connection stage)

Client article - connection phase

1. Introduction to the principle

2. Client logic

2.1 Connection phase

2.1.1 sc_server_start- Start Server

2.2.2 await_for_server- Waiting for connection state

2.2 Timing diagram

3. Summary

Guess you like

2.1.1 `sc_server_start`- Start Server

2.2.2 `await_for_server`- Waiting for connection state