Android WebRTC Complete Introductory Tutorial 01: Using the Camera

There is no official tutorial for WebRTC Android, not even API documentation. This is a strange thing, WebRTC was developed by Google after all. At present, the official documents and Demo are only available on the web side. Although the writing is simple and easy to understand, and the overall usage is the same as that of the Android side, there are still huge differences in the specific details.

Of course, you can still find some good tutorials on Google and Github. I will combine them here to form a complete introductory tutorial with a runnable demo.

First introduce some basic concepts
RTC (Real Time Communication): real-time communication
WebRTC: web-based real-time communication
Signaling: signaling, some strings describing media or networks
SDP (Session Description Protocol): session description protocol, mainly describing media information
ICE (Interactive Connectivity Establishment): Interactive connection establishment
STUN (Session Traversal Utilities for NAT): NAT session penetration tool
TURN (Traversal Using Relays around NAT): Relay penetration through NAT

Next is how to use

Add WebRTC library

Add dependencies in the module's build.gradle, which is the latest version of the official package (201901). Of course you can also build your own .

dependencies {
    
    
    ...
    implementation 'org.webrtc:google-webrtc:1.0.26131'
}

Add permissions

    <uses-permission android:name="android.permission.CAMERA" />
    <uses-permission android:name="android.permission.RECORD_AUDIO" />

Note that Android 6.0 and above need to open the camera and microphone permissions in the settings. Because it has nothing to do with the theme, so the application permission code is not added here.

Add SurfaceViewRenderer

It is a subclass of SurfaceView

<?xml version="1.0" encoding="utf-8"?>
<android.support.constraint.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context="cc.rome753.wat.MainActivity">

    <org.webrtc.SurfaceViewRenderer
        android:id="@+id/localView"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintTop_toTopOf="parent" />

</android.support.constraint.ConstraintLayout>

use the camera

The main steps are as follows

1. Create PeerConnectionFactory
2. Create and start VideoCapturer
3. Create VideoSource with PeerConnectionFactory
4. Create VideoTrack with PeerConnectionFactory and VideoSource
5. Initialize the video control SurfaceViewRenderer
6. Display VideoTrack to SurfaceViewRenderer

public class MainActivity extends AppCompatActivity {
    
    

    @Override
    protected void onCreate(Bundle savedInstanceState) {
    
    
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        // create PeerConnectionFactory
        PeerConnectionFactory.InitializationOptions initializationOptions =
                PeerConnectionFactory.InitializationOptions.builder(this).createInitializationOptions();
        PeerConnectionFactory.initialize(initializationOptions);
        PeerConnectionFactory peerConnectionFactory = PeerConnectionFactory.builder().createPeerConnectionFactory();

        // create AudioSource
        AudioSource audioSource = peerConnectionFactory.createAudioSource(new MediaConstraints());
        AudioTrack audioTrack = peerConnectionFactory.createAudioTrack("101", audioSource);

        EglBase.Context eglBaseContext = EglBase.create().getEglBaseContext();

        SurfaceTextureHelper surfaceTextureHelper = SurfaceTextureHelper.create("CaptureThread", eglBaseContext);
        // create VideoCapturer
        VideoCapturer videoCapturer = createCameraCapturer();
        VideoSource videoSource = peerConnectionFactory.createVideoSource(videoCapturer.isScreencast());
        videoCapturer.initialize(surfaceTextureHelper, getApplicationContext(), videoSource.getCapturerObserver());
        videoCapturer.startCapture(480, 640, 30);

        SurfaceViewRenderer localView = findViewById(R.id.localView);
        localView.setMirror(true);
        localView.init(eglBaseContext, null);

        // create VideoTrack
        VideoTrack videoTrack = peerConnectionFactory.createVideoTrack("101", videoSource);
        // display in localView
        videoTrack.addSink(localView);
    }

    private VideoCapturer createCameraCapturer() {
    
    
        Camera1Enumerator enumerator = new Camera1Enumerator(false);
        final String[] deviceNames = enumerator.getDeviceNames();

        // First, try to find front facing camera
        for (String deviceName : deviceNames) {
    
    
            if (enumerator.isFrontFacing(deviceName)) {
    
    
                VideoCapturer videoCapturer = enumerator.createCapturer(deviceName, null);

                if (videoCapturer != null) {
    
    
                    return videoCapturer;
                }
            }
        }

        // Front facing camera not found, try something else
        for (String deviceName : deviceNames) {
    
    
            if (!enumerator.isFrontFacing(deviceName)) {
    
    
                VideoCapturer videoCapturer = enumerator.createCapturer(deviceName, null);

                if (videoCapturer != null) {
    
    
                    return videoCapturer;
                }
            }
        }

        return null;
    }

}

appendix

https://webrtc.org/start/
https://codelabs.developers.google.com/codelabs/webrtc-web/#0
https://developer.mozilla.org/en-US/docs/Web/API/MediaStream
https://vivekc.xyz/getting-started-with-webrtc-for-android-daab1e268ff4
https://www.html5rocks.com/en/tutorials/webrtc/basics/

Reprint: https://www.jianshu.com/p/eb5fd116e6c8