一、背景
需要使用Camera2来实时抓取每一帧的图像
二、Camera2相关的类
1.CameraManager
摄像头管理类，可以获取摄像头ID、摄像头支持分辨率、传感器旋转角度等摄像头信息。
同时打开关闭摄像头也需要通过这个类
以下是这个类的基本使用，用来获取摄像头信息
private void getCameraInfo(){
CameraManager cameraManager= (CameraManager) getSystemService(Context.CAMERA_SERVICE);
try {
cameraNames = cameraManager.getCameraIdList();
} catch (CameraAccessException e) {
throw new RuntimeException(e);
}
for (String cameraName : cameraNames) {
StringBuilder info=new StringBuilder("cameraId:"+cameraName+"{\n");
List<int[]> list=new ArrayList<>();
CameraCharacteristics characteristics = null;
try {
characteristics = cameraManager.getCameraCharacteristics(cameraName);
} catch (CameraAccessException e) {
throw new RuntimeException(e);
}
int sensorOrientation = characteristics.get(CameraCharacteristics.SENSOR_ORIENTATION);
info.append("sensorOrientation:"+sensorOrientation+",\n");
Range<Integer>[] ranges = characteristics.get(CameraCharacteristics.CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES);
Log.e("lkx","range:"+ Arrays.toString(ranges));
int lensFacing = characteristics.get(CameraCharacteristics.LENS_FACING);
Long aLong = characteristics.get(CameraCharacteristics.SENSOR_INFO_MAX_FRAME_DURATION);
Log.e("lkx",cameraName+":"+aLong);
switch (lensFacing) {
case CameraMetadata.LENS_FACING_FRONT:
info.append("lensFacing(前后摄):"+"front"+",\n");
break;
case CameraMetadata.LENS_FACING_BACK:
info.append("lensFacing(前后摄):"+"back"+",\n");
break;
case CameraMetadata.LENS_FACING_EXTERNAL:
info.append("lensFacing(前后摄):"+"external"+",\n");
break;
}
info.append("}\n\n");
Size[] supportedSizes = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
.getOutputSizes(ImageFormat.YUV_420_888);
for (Size size : supportedSizes) {
int width = size.getWidth();
int height = size.getHeight();
// 在这里处理支持的分辨率格式
int[] temp={
width,height};
list.add(temp);
}
cameraInfoList.put(cameraName, String.valueOf(info));
map.put(cameraName,list);
}
}
要获取摄像机的参数信息，主要是通过CameraCharacteristics这个类的get方法：
CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES FPS范围
SENSOR_ORIENTATION 传感器旋转角度，有的摄像头不是0度，需要根据传感器角度去调整预览和拍照的角度
LENS_FACING 前后摄
SENSOR_INFO_MAX_FRAME_DURATION 两帧之间的间隔，单位是纳秒 使用1秒除以这个参数可以计算出帧率
SCALER_STREAM_CONFIGURATION_MAP 支持的分辨率
2.CameraDevice
用于表示连接的相机设备
该类中有一个方法
CaptureRequest.Builder createCaptureRequest(int templateType)
用于创建一个CaptureRequest
其中传入的参数需要注意：
TEMPLATE_PREVIEW : 创建预览的请求
TEMPLATE_STILL_CAPTURE: 创建一个适合于静态图像捕获的请求，图像质量优先于帧速率
TEMPLATE_RECORD : 创建视频录制的请求
TEMPLATE_VIDEO_SNAPSHOT : 创建视视频录制时截屏的请求
TEMPLATE_ZERO_SHUTTER_LAG :创建一个适用于零快门延迟的请求。在不影响预览帧率的情况下最大化图像质量
TEMPLATE_MANUAL : 创建一个基本捕获请求，这种请求中所有的自动控制都是禁用的(自动曝光，自动白平衡、自动焦点)
预览时需要传入TEMPLATE_PREVIEW
createCaptureSession
public abstract void createCaptureSession (List outputs,
CameraCaptureSession.StateCallback callback,
Handler handler)
outputs 目标Surface集
callback 创建CaptureSession的回调
handler 调用callback的线程
通过该方法创建一个CaptureSession
mCameraDevice.createCaptureSession(Arrays.asList(surface, imageReader.getSurface()),
new CameraCaptureSession.StateCallback() {
@Override
public void onConfigured(@NonNull CameraCaptureSession session) {
if (mCameraDevice == null) {
return;
}
cameraCaptureSession = session;
updatePreview();
}
@Override
public void onConfigureFailed(@NonNull CameraCaptureSession session) {
Toast.makeText(MainActivity.this, "Configuration change", Toast.LENGTH_SHORT).show();
}
}, null);
3.CaptureRequest
表示一次操作请求，预览、拍照时都需要传入这个对象。
通过上面CameraDevice的createCaptureRequest可以获得CaptureRequest.Builder
CaptureRequest.Builder.build()就可以获得这个对象
4.CaptureRequest.Builder
CaptureRequest的构建器 通过CameraDevice的createCaptureRequest获取实例
addTarget(Surface outputTarget)
添加一个Surface到列表中作为此请求的输出目标
可以添加多个surface，但是重复添加无效
扫描二维码关注公众号，回复：
17383501 查看本文章
set
set(Key key, T value)
为CaptureRequest的字段设置值
例：
// 自动聚焦
captureRequestBuilder.set(CaptureRequest.CONTROL_AF_MODE, CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE)
5.CameraCaptureSession
CameraDevice配置的一个捕获会话，调用此类的方法可以进行预览、拍照。
setRepeatingRequest(request: CaptureRequest, listener: CameraCaptureSession.CaptureCallback?, handler: Handler?)
mCaptureSession.setRepeatingRequest(mPreviewRequest,
mCaptureCallback, mBackgroundHandler);
...
private CameraCaptureSession.CaptureCallback mCaptureCallback
= new CameraCaptureSession.CaptureCallback() {
@Override
public void onCaptureProgressed(@NonNull CameraCaptureSession session,
@NonNull CaptureRequest request,
@NonNull CaptureResult partialResult) {
process(partialResult);
}
@Override
public void onCaptureCompleted(@NonNull CameraCaptureSession session,
@NonNull CaptureRequest request,
@NonNull TotalCaptureResult result) {
process(result);
}
};
该方法创建一个无限捕捉图像的请求，可以用于预览，方法需要一个request，可传入回调函数
6.ImageReader
ImageReader类允许应用程序直接访问渲染成表面的图像数据
mImageReader = ImageReader.newInstance(mWidth, mHeight,
ImageFormat.YUV_420_888, /*maxImages*/60);
mImageReader.setOnImageAvailableListener(
mOnImageAvailableListener, mBackgroundHandler);
//通过addTarget把imagereader的surface添加到预览的request里，这样可以实时获取每帧的图像数据
mPreviewRequestBuilder.addTarget(mImageReader.getSurface());
private final ImageReader.OnImageAvailableListener mOnImageAvailableListener
= new ImageReader.OnImageAvailableListener() {
@Override
public void onImageAvailable(ImageReader reader) {
//由于添加到了预览的request，每获取到一帧图像都会回调这个方法
Image mImage=reader.acquireNextImage();
//获取到的image对象就是图片数据，此处可以对图片进行操作
mImage.close(); //记得释放
}
};
三、Demo
此Demo仅仅实现最低需求的预览和拍照，代码需要完善。
逻辑代码
public class MainActivity extends AppCompatActivity {
private static final String TAG = "Camera2Demo";
private TextureView textureView;
private CameraDevice mCameraDevice;
private Size imageDimension;
private CaptureRequest.Builder captureRequestBuilder;
private CameraCaptureSession cameraCaptureSession;
private ImageReader imageReader;
private CaptureRequest captureRequest;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
initPermission(); //权限申请
textureView = findViewById(R.id.texture_view);
textureView.setSurfaceTextureListener(textureListener);
CameraCaptureSession.CaptureCallback CaptureCallback
= new CameraCaptureSession.CaptureCallback() {
@Override
public void onCaptureCompleted(@NonNull CameraCaptureSession session,
@NonNull CaptureRequest request,
@NonNull TotalCaptureResult result) {
//点击拍照后调用此回调 此处弹出提示
Toast.makeText(MainActivity.this,"Saved:xx/xx/xx/xx.jpg",Toast.LENGTH_SHORT).show();
}
};
findViewById(R.id.picture).setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
try {
//创建了一个拍照的请求
final CaptureRequest.Builder captureBuilder =
mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_STILL_CAPTURE);
//与imageReader关联，当点击拍照获取到可用图片时，会触发ImageReader的回调，可在ImageReader的回调中对图片进行保存等处理
captureBuilder.addTarget(imageReader.getSurface());
cameraCaptureSession.capture(captureBuilder.build(),CaptureCallback,null);
} catch (CameraAccessException e) {
throw new RuntimeException(e);
}
}
});
}
private final TextureView.SurfaceTextureListener textureListener =
new TextureView.SurfaceTextureListener() {
@Override
public void onSurfaceTextureAvailable(@NonNull SurfaceTexture surface, int width, int height) {
Log.e(TAG, "onSurfaceTextureAvailable");
openCamera();
}
@Override
public void onSurfaceTextureSizeChanged(@NonNull SurfaceTexture surface, int width, int height) {
Log.e(TAG, "onSurfaceTextureSizeChanged");
}
@Override
public boolean onSurfaceTextureDestroyed(@NonNull SurfaceTexture surface) {
Log.e(TAG, "onSurfaceTextureDestroyed");
return false;
}
@Override
public void onSurfaceTextureUpdated(@NonNull SurfaceTexture surface) {
Log.e(TAG, "onSurfaceTextureUpdated");
}
};
//相机状态变化时会调用这里的回调函数
private final CameraDevice.StateCallback stateCallback = new CameraDevice.StateCallback() {
@Override
public void onOpened(@NonNull CameraDevice camera) {
//相机打开时执行
Log.e(TAG, "onOpened");
mCameraDevice=camera;
//创建相机预览会话
createCameraPreviewSession();
}
@Override
public void onDisconnected(@NonNull CameraDevice camera) {
//相机链接断开
}
@Override
public void onError(@NonNull CameraDevice camera, int error) {
}
};
private void openCamera() {
CameraManager cameraManager = (CameraManager) getSystemService(Context.CAMERA_SERVICE);
try {
//通过cameraId获取Camera参数
String cameraId = cameraManager.getCameraIdList()[0];
CameraCharacteristics characteristics = cameraManager.getCameraCharacteristics(cameraId);
StreamConfigurationMap map = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
imageDimension=map.getOutputSizes(SurfaceTexture.class)[0];
imageReader=ImageReader.newInstance(640,480, ImageFormat.YUV_420_888,10);
imageReader.setOnImageAvailableListener(onImageAvailableListener,null);
if (ActivityCompat.checkSelfPermission(this, android.Manifest.permission.CAMERA) != PackageManager.PERMISSION_GRANTED) {
// TODO: Consider calling
// ActivityCompat#requestPermissions
// here to request the missing permissions, and then overriding
// public void onRequestPermissionsResult(int requestCode, String[] permissions,
// int[] grantResults)
// to handle the case where the user grants the permission. See the documentation
// for ActivityCompat#requestPermissions for more details.
return;
}
cameraManager.openCamera(cameraId, stateCallback, null);
} catch (CameraAccessException e) {
throw new RuntimeException(e);
}
}
private boolean initPermission(){
ActivityCompat.requestPermissions(MainActivity.this, new String[]{
android.Manifest.permission.READ_EXTERNAL_STORAGE, android.Manifest.permission.WRITE_EXTERNAL_STORAGE, android.Manifest.permission.CAMERA}, 1);
// 高版本Android SDK时使用如下代码
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.R) {
if(!Environment.isExternalStorageManager()){
Intent intent = new Intent(Settings.ACTION_MANAGE_ALL_FILES_ACCESS_PERMISSION);
startActivity(intent);
return false;
}
}
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
if (checkSelfPermission(android.Manifest.permission.CAMERA) != PackageManager.PERMISSION_GRANTED) {
requestPermissions(new String[] {
Manifest.permission.CAMERA}, 1);
return false;
}
}
return true;
}
private void createCameraPreviewSession(){
SurfaceTexture surfaceTexture=textureView.getSurfaceTexture();
assert surfaceTexture!=null;
surfaceTexture.setDefaultBufferSize(imageDimension.getWidth(),imageDimension.getHeight());
//预览的输出画面
Surface surface=new Surface(surfaceTexture);
try {
//预览请求
captureRequestBuilder=mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
captureRequestBuilder.addTarget(surface);
mCameraDevice.createCaptureSession(Arrays.asList(surface, imageReader.getSurface()),
new CameraCaptureSession.StateCallback() {
@Override
public void onConfigured(@NonNull CameraCaptureSession session) {
if (mCameraDevice == null) {
return;
}
cameraCaptureSession = session;
updatePreview();
}
@Override
public void onConfigureFailed(@NonNull CameraCaptureSession session) {
Toast.makeText(MainActivity.this, "Configuration change", Toast.LENGTH_SHORT).show();
}
}, null);
} catch (CameraAccessException e) {
throw new RuntimeException(e);
}
}
private void updatePreview(){
captureRequestBuilder.set(CaptureRequest.CONTROL_AF_MODE,CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE);
captureRequest = captureRequestBuilder.build();
try {
cameraCaptureSession.setRepeatingRequest(captureRequest,null,null);
} catch (CameraAccessException e) {
throw new RuntimeException(e);
}
}
private final ImageReader.OnImageAvailableListener onImageAvailableListener=
new ImageReader.OnImageAvailableListener() {
@Override
public void onImageAvailable(ImageReader reader) {
Image image=null;
image=reader.acquireLatestImage();
Log.e("lkx","1");
//此处可对图片进行处理 比如保存到本地
image.close();
}
};
}
布局文件
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MainActivity"
android:orientation="vertical">
<TextureView
android:id="@+id/texture_view"
android:layout_width="match_parent"
android:layout_height="588dp"></TextureView>
<Button
android:id="@+id/picture"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_gravity="center"
android:text="take_photo" />
</LinearLayout>
效果展示
启动apk，是一个预览框和一个拍照按钮
点击拍照，会弹出提示框
log也打印了，imageReader的回调正常调用
四、思考
对于刚开始提出来的需求，需要对每一帧图像进行实时抓取。通过这个demo我们不难发现，imageReader是通过addTarget添加到了拍照这个请求上，每次点击拍照，会生成一个可用图像，有了可用对象才会触发imageReader的回调。既然这样我们可以将imageReader通过addTarget绑定到预览的那个请求上，预览是无限获取每一帧的图像，每一帧都是一个可用图像，那么每一帧就都可以触发imageReader的回调函数，这样就可以在回调函数中对每一帧的图像进行处理。
需要注意，获取图像的帧率受很多因素影响，图片过大会使帧数降低，使用使应该尽量保证图片尺寸较小。初始化ImageReader时，newInstance方法的最后一个参数maxImage如果太小也会影响帧数。同时，也应该尽量保证对每一帧的图像数据处理耗时较低，可以通过Handler交给其他线程完成耗时任务。
The maxImages parameter determines the maximum number of Image objects that can be be acquired from the ImageReader simultaneously. Requesting more buffers will use up more memory, so it is important to use only the minimum number necessary for the use case.
上面是谷歌对maxImages这个参数的解释，翻译为中文：
maxImages参数决定可以同时从ImageReader获取的图像对象的最大数量。请求更多缓冲区将消耗更多内存，因此仅使用用例所需的最小数量是很重要的。
参数小会影响帧率，参数大会消耗更多内存。这两者之间需要进行平衡。