Realize Android Camera2 camera preview in ten minutes

1 Introduction

Because it needs to be used at work Android Camera2 API, but because Camera2it is more complicated and the online information is messy, there is a certain entry threshold, so it took a few days to systematically study it and CSDNrecord it on the website, hoping to help more friends.

2. Overview of Camera2 API

Camera2 APIThe package name is android.hardware.camera2, which is Android 5.0a set of interfaces for calling camera devices launched later to replace the original ones Camera. Camera2 APIThe pipeline design is used to make the data flow from the camera to the camera Surface. Camera2 APIWhen using the function of taking pictures and recording videos, it mainly involves the following categories:

CameraManager: CameraThe management class of the device, through which the Cameradevice information of the device can be queried and CameraDevicethe object can be obtained
CameraDevice: CameraDeviceProvides Cameraa series of fixed parameters related to the device, such as basic settings and output formats. This information is contained in CameraCharacteristicthe class, which can be obtained through getCameraCharacteristics(String)the object of this class.
CaptureSession: In Camera API, how to Cameraobtain video or picture stream from the device, first need to use the output Surfaceand CameraDevicecreate aCameraCaptureSession
CaptureRequest: This class defines Camerathe parameters required by a device to obtain frame data, which can be CameraDevicecreated through the factory method Request Builderto obtainCaptureRequest
CaptureResult: When a request is processed, an TotalCaptureResultobject will be returned, which contains the parameters used by Camerathe device to execute this time and its own state.Request

A Androiddevice can have multiple cameras. Each camera is a camera device, and a camera device can output multiple streams simultaneously.
insert image description here

3. Pre-settings

3.1 Add permissions

AndroidManifest.xmlDeclare permissions in

<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.CAMERA" />

3.2 Application permissions

ActivityCompat.requestPermissions(
        this@MainActivity,
        arrayOf(
            Manifest.permission.CAMERA,
            Manifest.permission.WRITE_EXTERNAL_STORAGE,
            Manifest.permission.RECORD_AUDIO
        ), 123
    )

4. Get camera list

4.1 Get camera list

Obtaining the camera list needs to be used , and the camera list can be obtained CameraManagerthroughcameraManager.cameraIdList

private val cameraManager =
        context.getSystemService(Context.CAMERA_SERVICE) as CameraManager

// 获取所有摄像头的CameraID
fun getCameraIds(): Array<String> {
    
    
    return cameraManager.cameraIdList
}

4.2 Judging the front/rear camera

Through this method, the orientation of the camera can be obtained to determine whether it is a front camera or a rear camera.

/**
* 获取摄像头方向
*/
fun getCameraOrientationString(cameraId: String): String {
    
    
   val characteristics = cameraManager.getCameraCharacteristics(cameraId)
   val lensFacing = characteristics.get(CameraCharacteristics.LENS_FACING)!!
   return when (lensFacing) {
    
    
       CameraCharacteristics.LENS_FACING_BACK -> "后摄(Back)"
       CameraCharacteristics.LENS_FACING_FRONT -> "前摄(Front)"
       CameraCharacteristics.LENS_FACING_EXTERNAL -> "外置(External)"
       else -> "Unknown"
   }
}

There is also a simple way of judging, under normal circumstances, cameraIdit 0is a postactive, cameraIdand 1it is a proactive.

4.3 Get it and try it

let's get it

val cameraIds = viewModel.getCameraIds()
cameraIds.forEach{
    
     cameraId ->
    val orientation = viewModel.getCameraOrientationString(cameraId)
    Log.i(TAG,"cameraId : $cameraId - $orientation")
}

After running, you can find that the log is printed

cameraId : 0 - 后摄(Back)
cameraId : 1 - 前摄(Front)

5. Realize camera preview

5.1 Modify the layout

to modify XMLthe layout

<?xml version="1.0" encoding="utf-8"?>
<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:background="@color/black"
    xmlns:app="http://schemas.android.com/apk/res-auto">

    <SurfaceView
        android:id="@+id/surface_view"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintTop_toTopOf="parent"
        app:layout_constraintBottom_toBottomOf="parent" />

    <Button
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintBottom_toBottomOf="parent"
        android:id="@+id/btn_take_picture"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_gravity="bottom|center"
        android:layout_marginBottom="64dp"
        android:text="拍照"/>

</FrameLayout>

5.2 Declare camera parameters and member variables

//后摄 : 0 ，前摄 : 1
private val cameraId = "0"
private val TAG = CameraActivity::class.java.simpleName
private lateinit var cameraDevice: CameraDevice
private val cameraThread = HandlerThread("CameraThread").apply {
    
     start() }
private val cameraHandler = Handler(cameraThread.looper)
private val cameraManager: CameraManager by lazy {
    
    
    getSystemService(Context.CAMERA_SERVICE) as CameraManager
}
private val characteristics: CameraCharacteristics by lazy {
    
    
    cameraManager.getCameraCharacteristics(cameraId)
}
private lateinit var session: CameraCaptureSession

5.3 Add SurfaceView callback

Add SurfaceViewa callback and SurfaceViewinitialize the camera when creating

override fun onCreate(savedInstanceState: Bundle?) {
    
    
	super.onCreate(savedInstanceState)
	binding = ActivityCameraBinding.inflate(layoutInflater)
	setContentView(binding.root)
	binding.surfaceView.holder.addCallback(object : SurfaceHolder.Callback {
    
    
	    override fun surfaceChanged(holder: SurfaceHolder,format: Int, width: Int,height: Int) = Unit
	
	    override fun surfaceDestroyed(holder: SurfaceHolder) = Unit
	
	    override fun surfaceCreated(holder: SurfaceHolder) {
    
    
		    //为了确保设置了大小，需要在主线程中初始化camera
	        binding.root.post {
    
    
	             openCamera(cameraId)
	        }
	    }
	})
}

5.4 Turn on the camera

@SuppressLint("MissingPermission")
private fun openCamera(cameraId: String) {
    
    
    cameraManager.openCamera(cameraId, object : CameraDevice.StateCallback() {
    
    
        override fun onOpened(camera: CameraDevice) {
    
    
			cameraDevice = camera
			startPreview()
        }

        override fun onDisconnected(camera: CameraDevice) {
    
    
            this@CameraActivity.finish()
        }

        override fun onError(camera: CameraDevice, error: Int) {
    
    
            Toast.makeText(application, "openCamera Failed:$error", Toast.LENGTH_SHORT).show()
        }
    }, cameraHandler)
}

5.5 Start preview

private fun startPreview() {
    
    
	//因为摄像头设备可以同时输出多个流，所以可以传入多个surface
    val targets = listOf(binding.surfaceView.holder.surface /*,这里可以传入多个surface*/)
    cameraDevice.createCaptureSession(targets, object : CameraCaptureSession.StateCallback() {
    
    
        override fun onConfigured(captureSession: CameraCaptureSession) {
    
    
        	//赋值session
            session = captureSession

            val captureRequest = cameraDevice.createCaptureRequest(
                CameraDevice.TEMPLATE_PREVIEW
            ).apply {
    
     addTarget(binding.surfaceView.holder.surface) }

            //这将不断地实时发送视频流，直到会话断开或调用session.stoprepeat()
            session.setRepeatingRequest(captureRequest.build(), null, cameraHandler)
        }

        override fun onConfigureFailed(session: CameraCaptureSession) {
    
    
            Toast.makeText(application,"session configuration failed",Toast.LENGTH_SHORT).show()
        }
    }, cameraHandler)
}

5.6 Let's see the effect

You can see that the preview screen is out, but the proportion is wrong, and there is stretching and deformation. Next, we will solve this problem

insert image description here

5.7 Correction of tensile deformation

5.7.1 Create a new AutoFitSurfaceView

Newly AutoFitSurfaceViewinherited from SurfaceView, this class can be adjusted to the aspect ratio we specify, and center cropped when displaying the screen.



class AutoFitSurfaceView @JvmOverloads constructor(
    context: Context,
    attrs: AttributeSet? = null,
    defStyle: Int = 0
) : SurfaceView(context, attrs, defStyle) {
    
    

    private var aspectRatio = 0f

    /**
     * 设置此视图的宽高比。视图的大小将基于从参数中计算的比率来测量。
     *
     * @param width  相机水平分辨率
     * @param height 相机垂直分辨率
     */
    fun setAspectRatio(width: Int, height: Int) {
    
    
        require(width > 0 && height > 0) {
    
     "Size cannot be negative" }
        aspectRatio = width.toFloat() / height.toFloat()
        holder.setFixedSize(width, height)
        requestLayout()
    }

    override fun onMeasure(widthMeasureSpec: Int, heightMeasureSpec: Int) {
    
    
        super.onMeasure(widthMeasureSpec, heightMeasureSpec)
        val width = MeasureSpec.getSize(widthMeasureSpec)
        val height = MeasureSpec.getSize(heightMeasureSpec)
        if (aspectRatio == 0f) {
    
    
            setMeasuredDimension(width, height)
        } else {
    
    

            // Performs center-crop transformation of the camera frames
            val newWidth: Int
            val newHeight: Int
            val actualRatio = if (width > height) aspectRatio else 1f / aspectRatio
            if (width < height * actualRatio) {
    
    
                newHeight = height
                newWidth = (height * actualRatio).roundToInt()
            } else {
    
    
                newWidth = width
                newHeight = (width / actualRatio).roundToInt()
            }

            Log.d(TAG, "Measured dimensions set: $newWidth x $newHeight")
            setMeasuredDimension(newWidth, newHeight)
        }
    }

    companion object {
    
    
        private val TAG = AutoFitSurfaceView::class.java.simpleName
    }
}

5.7.2 `XML`The layout will `SurfaceView`be replaced by`AutoFitSurfaceView`

<?xml version="1.0" encoding="utf-8"?>
<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:background="@color/black"
    xmlns:app="http://schemas.android.com/apk/res-auto">

    <!--<SurfaceView-->
    <com.heiko.mycamera2test.AutoFitSurfaceView
        android:id="@+id/surface_view"
        android:layout_width="match_parent"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintTop_toTopOf="parent"
        app:layout_constraintBottom_toBottomOf="parent"
        android:layout_height="match_parent" />

    <Button
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintBottom_toBottomOf="parent"
        android:id="@+id/btn_take_picture"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_gravity="bottom|center"
        android:layout_marginBottom="64dp"
        android:text="拍照"/>

</FrameLayout>

Note that the root layout cannot be used here ConstraintLayout, otherwise there will still be problems with the aspect ratio

5.7.3 Get the maximum supported preview size

Create a new class, this class matches the maximum supported preview size SmartSizeby comparing the displayed resolution with the resolution supported by the cameraSurfaceView

import android.graphics.Point
import android.hardware.camera2.CameraCharacteristics
import android.hardware.camera2.params.StreamConfigurationMap
import android.util.Size
import android.view.Display
import java.lang.Math.max
import java.lang.Math.min

/** Helper class used to pre-compute shortest and longest sides of a [Size] */
class SmartSize(width: Int, height: Int) {
    
    
    var size = Size(width, height)
    var long = max(size.width, size.height)
    var short = min(size.width, size.height)
    override fun toString() = "SmartSize(${
      
      long}x${
      
      short})"
}

/** Standard High Definition size for pictures and video */
val SIZE_1080P: SmartSize = SmartSize(1920, 1080)

/** Returns a [SmartSize] object for the given [Display] */
fun getDisplaySmartSize(display: Display): SmartSize {
    
    
    val outPoint = Point()
    display.getRealSize(outPoint)
    return SmartSize(outPoint.x, outPoint.y)
}

/**
 * Returns the largest available PREVIEW size. For more information, see:
 * https://d.android.com/reference/android/hardware/camera2/CameraDevice and
 * https://developer.android.com/reference/android/hardware/camera2/params/StreamConfigurationMap
 */
fun <T>getPreviewOutputSize(
        display: Display,
        characteristics: CameraCharacteristics,
        targetClass: Class<T>,
        format: Int? = null
): Size {
    
    

    // Find which is smaller: screen or 1080p
    val screenSize = getDisplaySmartSize(display)
    val hdScreen = screenSize.long >= SIZE_1080P.long || screenSize.short >= SIZE_1080P.short
    val maxSize = if (hdScreen) SIZE_1080P else screenSize

    // If image format is provided, use it to determine supported sizes; else use target class
    val config = characteristics.get(
            CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)!!
    if (format == null)
        assert(StreamConfigurationMap.isOutputSupportedFor(targetClass))
    else
        assert(config.isOutputSupportedFor(format))
    val allSizes = if (format == null)
        config.getOutputSizes(targetClass) else config.getOutputSizes(format)

    // Get available sizes and sort them by area from largest to smallest
    val validSizes = allSizes
            .sortedWith(compareBy {
    
     it.height * it.width })
            .map {
    
     SmartSize(it.width, it.height) }.reversed()

    // Then, get the largest output size that is smaller or equal than our max size
    return validSizes.first {
    
     it.long <= maxSize.long && it.short <= maxSize.short }.size
}

5.7.4 Setting the aspect ratio

Before we call the method openCamera(), we first set the aspect ratiosetAspectRatio()

binding.surfaceView.holder.addCallback(object : SurfaceHolder.Callback {
    
    
    //...省略了代码....
    override fun surfaceCreated(holder: SurfaceHolder) {
    
    
	    //设置宽高比
	    setAspectRatio()
        //为了确保设置了大小，需要在主线程中初始化camera
        binding.root.post {
    
    
            openCamera2(cameraId)
        }
    }
})
	
private fun setAspectRatio() {
    
    
	val previewSize = getPreviewOutputSize(
	    binding.surfaceView.display,
	    characteristics,
	    SurfaceHolder::class.java
	)
	Log.d(TAG, "Selected preview size: $previewSize")
	binding.surfaceView.setAspectRatio(previewSize.width, previewSize.height)
}

5.7.5 Run preview again

As you can see, the scale is now normal
insert image description here

5.8 Destroy the camera

When Activitydestroying, we also want to destroy the camera, the code is as follows

override fun onStop() {
    
    
    super.onStop()
    try {
    
    
        cameraDevice.close()
    } catch (exc: Throwable) {
    
    
        Log.e(TAG, "Error closing camera", exc)
    }
}

override fun onDestroy() {
    
    
    super.onDestroy()
    cameraThread.quitSafely()
    //imageReaderThread.quitSafely()
}

6. Other