Scrcpy Source Code Analysis Series
[Screencasting] Scrcpy Source Code Analysis 1 (Compilation)
[Screencasting] Scrcpy Source Code Analysis 2 (Client - Connection Stage)
[Screencasting] Scrcpy Source Code Analysis 3 (Client - Screencasting Stage)
[Screencasting] 】Srcpy source code analysis four (final chapter - Server chapter)
In the first two articles, we explored the connection and projection logic of Scrcpy Client. In this article, we will continue to explore the logic of the server side.
1. Entry function
Let's recall first, do you still remember how the server side works?
Answer: The client executes
adb pushto upload the server program to the device side, and then executesapp_processthe server program to run. The full command isadb -s serial shell CLASSPATH=/data/local/tmp/scrcpy-server.jar app_process / com.genymobile.scrcpy.Server 1.25 [PARAMS].
app_processOne of the advantages is that it is convenient for us to run a pure java program on the Android side (it is the bytecode of dalvik, not jvm bytecode), and the other is to escalate the privileges, so that the program has root privileges or shell equivalent privileges.
Because the class specified by the Client is com.genymobile.scrcpy.Server, the entry method of the Server is the method Server.javaof the class main(), and its key code is:
// Server.java
public static void main(String... args) {
// 解析参数
Options options = createOptions(args);
// scrcpy方法
scrcpy(options);
}
private static void scrcpy(Options options) {
// 调用DesktopConnection的open函数
DesktopConnection connection = DesktopConnection.open(tunnelForward, control, sendDummyByte);
// 控制逻辑
Controller controller = new Controller(device, connection,);
startController(controller);
// 投屏逻辑
ScreenEncoder screenEncoder = new ScreenEncoder();
screenEncoder.streamScreen(device, connection.getVideoFd());
}
We can see that the main logic in the entry function is:
-
createOptions- Parse parameters. -
DesktopConnection.open- Connect to the PC terminal (as mentioned in the second article, so in business, the Android device is the Server, and the PC is the Client, but at the network level, the Client of the Android device, and the PC is the Server):// DeskopConnection.java private static final String SOCKET_NAME = "scrcpy"; public static DesktopConnection open(boolean tunnelForward, boolean control, boolean sendDummyByte) { videoSocket = connect(SOCKET_NAME); controlSocket = connect(SOCKET_NAME); return new DesktopConnection(videoSocket, controlSocket); } private static LocalSocket connect(String abstractName) { LocalSocket localSocket = new LocalSocket(); localSocket.connect(new LocalSocketAddress(abstractName)); return localSocket; }localabstract:scrcpyBecause the port mapping is enabled on the PC side through adb , you can connect to the PC by passingLocalServerSocketor specifying the Unix Socket Name here. The Unix Socket Name here is "scrcpy", which must be consistent with the one specified by adb.LocalSocketWith the logic on the PC side, you need to connect twice here to get videoSocket and controlSocket. At the same time, because these two are LocalSocket based on Unix Domain Socket, you can directly get their corresponding file descriptors, and you can directly read and write files laterFileDescription. Descriptor for network data transmission. Students who do not understand this part can review the logical description of this part of the second article Client. -
startController- Event control related logic, based on controlSocket. -
streamScreen- Screen projection related logic, based on videoSocket.
Seeing this, we should know that after the Sever program starts up, it will connect to the PC and get two Sockets.
Next, let's continue to look at the projection and control logic.
2. Screen projection logic
The entry point of the projection logic is streamScreenthe method:
// ScreenEncoder.java
public void streamScreen(Device device, FileDescriptor fd) {
internalStreamScreen(device, fd);
}
private void internalStreamScreen(Device device, FileDescriptor fd) {
// MediaCodec录屏的模板代码
MediaFormat format = createFormat(bitRate, maxFps, codecOptions);
MediaCodec codec = createCodec(encoderName);
IBinder display = createDisplay();
surface = codec.createInputSurface();
setDisplaySurface(display, surface, videoRotation, contentRect, unlockedVideoRect, layerStack);
codec.start();
// 编码
encode(codec, fd);
}
private boolean encode(MediaCodec codec, FileDescriptor fd) {
while (!consumeRotationChange() && !eof) {
int outputBufferId = codec.dequeueOutputBuffer(bufferInfo, -1);
ByteBuffer codecBuffer = codec.getOutputBuffer(outputBufferId);
// 写fd即发送给PC侧
IO.writeFully(fd, codecBuffer);
}
}
We have seen that the part of screencasting is actually using screen recording and MediaCodechardcoding. This part of partial template code is basically the set MediaCodecparameters, get the H264 packet data through hard coding, and then IO.writeFullysend the data by writing to fd.
Under the general flow chart:
3. Control logic
The entry point of the control logic is startControllerthe method:
private static Thread startController(final Controller controller) {
Thread thread = new Thread(new Runnable() {
@Override
public void run() {
controller.control();
}
});
thread.start();
}
public void control() {
while (true) {
handleEvent();
}
}
private void handleEvent() {
// 从controlSocket的inputStream读数据
ControlMessage msg = connection.receiveControlMessage();
switch (msg.getType()) {
case ControlMessage.TYPE_INJECT_KEYCODE:
injectKeycode(msg.getAction(), msg.getKeycode(), msg.getRepeat(), msg.getMetaState());
case ControlMessage.TYPE_INJECT_TOUCH_EVENT:
injectTouch(msg.getAction(), msg.getPointerId(), msg.getPosition(), msg.getPressure(), msg.getButtons());
// ...
}
}
We see that the control part is to open the child thread, continuously read the control event data from the PC in the controlSocket, and then do different processing according to the event type. Here we see that keyboard events or mouse events are ultimately called to the ```injectXXX`` method. In fact, we can also guess that it must be to convert the events from the PC into Android events, and then distribute the events. So how does Scrcpy achieve this step?
3.1 Event injection
Let's look at the method first injectKeyCode:
// Controller.java
private boolean injectKeycode(int action, int keycode, int repeat, int metaState) {
// 调用Device的injectKeycode方法
device.injectKeyEvent(action, keycode, repeat, metaState, Device.INJECT_MODE_ASYNC);
}
// Device.java
public static boolean injectKeyEvent(int action, int keyCode, int repeat, int metaState, int displayId, int injectMode) {
long now = SystemClock.uptimeMillis();
// 构建一个KeyEvent
KeyEvent event = new KeyEvent(now, now, action, keyCode, repeat, metaState, KeyCharacterMap.VIRTUAL_KEYBOARD, 0, 0,
InputDevice.SOURCE_KEYBOARD);
return injectEvent(event, displayId, injectMode);
}
public static boolean injectEvent(InputEvent inputEvent, int displayId, int injectMode) {
InputManager.setDisplayId(inputEvent, displayId)
return ServiceManager.getInputManager().injectInputEvent(inputEvent, injectMode);
}
injectKeyCodeOne is built in the call chain KeyEvent, and then the last two methods are called:
-
InputManager.setDisplayId()- Specify the target Display for the event through the method called byInputEventreflectionsetDisplayMethod:// InputManager.java public static boolean setDisplayId(InputEvent inputEvent, int displayId) { Method method = getSetDisplayIdMethod(); method.invoke(inputEvent, displayId); return true; } private static Method getSetDisplayIdMethod() throws NoSuchMethodException { if (setDisplayIdMethod == null) { setDisplayIdMethod = InputEvent.class.getMethod("setDisplayId", int.class); } return setDisplayIdMethod; } -
ServiceManager.getInputManager().injectInputEvent()- Obtain the instance in the system through reflection , and wrap itInputManagerwith the class in the project :InputManager// ServiceManager.java public static InputManager getInputManager() { if (inputManager == null) { try { // 反射调用系统InputManager的getInstance方法 Method getInstanceMethod = android.hardware.input.InputManager.class.getDeclaredMethod("getInstance"); android.hardware.input.InputManager im = (android.hardware.input.InputManager) getInstanceMethod.invoke(null); // 将系统的InputManager实例传入工程自己的InputManager类，包装一下 inputManager = new InputManager(im); } catch (NoSuchMethodException | IllegalAccessException | InvocationTargetException e) { throw new AssertionError(e); } } return inputManager; }InputManagerThen call the method of the system through reflectioninjectInputEventto perform event injection processing, that is,InputManagerServicethe event is sent to the target Display through the system:private Method getInjectInputEventMethod() throws NoSuchMethodException { injectInputEventMethod = manager.getClass().getMethod("injectInputEvent", InputEvent.class, int.class); return injectInputEventMethod; } public boolean injectInputEvent(InputEvent inputEvent, int mode) { Method method = getInjectInputEventMethod(); return (boolean) method.invoke(manager, inputEvent, mode); }
injectTouchThe method is similar, the injection is MotionEvent. But Android MotionEventand both KeyEventare inherited from InputEvent, so in the end, injectInputEventthe event is sent to the target Display.
So our flowchart can be filled completely:
So far, the connection, screen projection, and control logic on the server side have been analyzed.
4. Timing diagram
As usual, a timing diagram is attached, and different colors represent different threads.
5. Summary
In this article, we explored the logic of the Scrcpy Server side. Compared with the Client side, the logic of the Server side is clearer and simpler. The points involved are Android screen recording, LocalSocket, MediaCode hardcoding, and event injection.
At this point, we have finished all the analysis of the Scrcpy software. We started from the project structure, studied its compilation system Meson, then went to the client side (PC side) to establish a connection and screen projection process, and finally to the server side (Android side) Connect, cast and control the process. The main process is still relatively clear.
In fact, there are three places that I personally feel most rewarded:
- ADB port mapping, this method provides convenience for mutual access between PC and mobile phone, combined with Unix Domain Socket, greatly expands the usage scenarios, and has a wide range of applications.
- SDL, I didn't know much about SDL before, I only know that it can be used as a multimedia-related interface. However, SDL library functions are widely used in Scrcpy to compare synchronization, event mechanism and other functions that are not related to multimedia. It can be said to be a powerful tool library. So now the author has decisively added SDL to his follow-up learning list.
- Android event injection, the client-side event injection mechanism is mainly used
InputEventprivate API,setDisplayandinjectInputEvent. In this way, you can build it yourselfKeyEventorMotionEventsend it to the specified screen later. It just so happens that the author is working on a project related to multi-screen recently, and there is a technical difficulty that needs to be overcome, that is, the user's touch events on the real physical screen are actually forwarded to a screen created by ourselvesVirtualDisplay. So I borrowed the method of event injection in Scrcpy, set the Display of the Event event to the ID of the VirtualDisplay, and then realized the forwarding through event injection.
Therefore, it is good to study successful open source software if you have nothing to do~