The architecture of the Camera is consistent with the overall architecture of the Android system, as shown in the figure below. This article mainly explains it from the following four aspects.
- Framework：Camera.java
- Android Runtime：android_hardware_Camera.cpp
- Library：Camera Client和Camera Service
- HAL：CameraHardwareInterface
一、Framework：Camera.java
Camera is a class directly used by the application layer software, covering all interfaces for operating the camera such as startup, preview, shooting and closing. The path of Camera.java in the Android source code is: framework/base/core/java/android/hardware. In order to illustrate the architecture of the entire Camera system, we will not analyze the functions of Camera.java horizontally here. Let's start with the open() method:
/**
* Creates a new Camera object to access the first back-facing camera on the
* device. If the device does not have a back-facing camera, this returns
* null.
* @see #open(int)
*/
public static Camera open() {
int numberOfCameras = getNumberOfCameras();
CameraInfo cameraInfo = new CameraInfo();
for (int i = 0; i < numberOfCameras; i++) {
getCameraInfo(i, cameraInfo);
if (cameraInfo.facing == CameraInfo.CAMERA_FACING_BACK) {
return new Camera(i);
}
}
return null;
}The open() method needs to pay attention to the following points:
- getNumberOfCameras is a native method, implemented in android_hardware_Camera.cpp;
- CameraInfo is a static internal class defined by Camera, including facing, orientation, canDisableShutterSound;
- getCameraInfo internally calls the native method _getCameraInfo to obtain camera information;
- open() starts the rear camera (CAMERA_FACING_BACK) by default.
Camera(int cameraId) {
mShutterCallback = null;
mRawImageCallback = null;
mJpegCallback = null;
mPreviewCallback = null;
mPostviewCallback = null;
mZoomListener = null;
Looper looper;
if ((looper = Looper.myLooper()) != null) {
mEventHandler = new EventHandler(this, looper);
} else if ((looper = Looper.getMainLooper()) != null) {
mEventHandler = new EventHandler(this, looper);
} else {
mEventHandler = null;
}
String packageName = ActivityThread.currentPackageName();
native_setup(new WeakReference<Camera>(this), cameraId, packageName);
}
Call the local method native_setup(), thus entering android_hardware_Camera.cpp ( frameworks\base\core\jni )
private native final void native_setup(Object camera_this, int cameraId,
String packageName);
二、Android Runtime：android_hardware_Camera.cpp
native_setup() is dynamically registered to JNI, and the android_hardware_Camera_native_setup() method is called through JNI.
// connect to camera service
static void android_hardware_Camera_native_setup(JNIEnv *env, jobject thiz,
jobject weak_this, jint cameraId, jstring clientPackageName)
{
// Convert jstring to String16
const char16_t *rawClientName = env->GetStringChars(clientPackageName, NULL);
jsize rawClientNameLen = env->GetStringLength(clientPackageName);
String16 clientName(rawClientName, rawClientNameLen);
env->ReleaseStringChars(clientPackageName, rawClientName);
sp<Camera> camera = Camera::connect(cameraId, clientName,
Camera::USE_CALLING_UID);
if (camera == NULL) {
jniThrowRuntimeException(env, "Fail to connect to camera service");
return;
}
// make sure camera hardware is alive
if (camera->getStatus() != NO_ERROR) {
jniThrowRuntimeException(env, "Camera initialization failed");
return;
}
jclass clazz = env->GetObjectClass(thiz);
if (clazz == NULL) {
jniThrowRuntimeException(env, "Can't find android/hardware/Camera");
return;
}
// We use a weak reference so the Camera object can be garbage collected.
// The reference is only used as a proxy for callbacks.
sp<JNICameraContext> context = new JNICameraContext(env, weak_this, clazz, camera);
context->incStrong((void*)android_hardware_Camera_native_setup);
camera->setListener(context);
// save context in opaque field
env->SetIntField(thiz, fields.context, (int)context.get());
}The android_hardware_Camera_native_setup() method requests to connect to the CameraService service by calling the Camera::connect() method. Enter the parameters:
- clientName is obtained by converting clientPackageName from jstring to String16 format;
- Camera::USE_CALLING_UID is an enumeration type defined in Camera.h, and its value is ICameraService::USE_CALLING_UID (also an enumeration type, with a value of -1).
Camera::connect() is located in Camera.cpp (frameworks\av\camera), thus entering the Library layer.
3. Library: Camera Client and Camera Service
As shown in the above architecture diagram, the three classes ICameraService.h, ICameraClient.h and ICamera.h (frameworks\av\include\camera) define the interface and architecture of the Camera, and the two files ICameraService.cpp and Camera.cpp use Based on the implementation of the Camera architecture, the specific functions of the Camera are realized by calling hardware-related interfaces at the lower layer. Camera.h is the interface of the Camera system to the upper layer.
Specifically, the Camera class inherits the template class CameraBase, and Camera::connect() calls the connect() method in CameraBase.cpp.
sp<Camera> Camera::connect(int cameraId, const String16& clientPackageName,
int clientUid)
{
return CameraBaseT::connect(cameraId, clientPackageName, clientUid);
}CameraBase actually inherits the DeathRecipient internal class of IBinder, and DeathRecipient virtually inherits from RefBase. RefBase is the basic class of reference counting in Android, which defines incStrong, decStrong, incWeak and decWeak and other pointer manipulation functions involving sp/wp. Of course, this is far away.
template <typename TCam>
struct CameraTraits {
};
template <typename TCam, typename TCamTraits = CameraTraits<TCam> >
class CameraBase : public IBinder::DeathRecipient
{
public:
static sp<TCam> connect(int cameraId,
const String16& clientPackageName,
int clientUid);
……
}
class DeathRecipient : public virtual RefBase { public: virtual void binderDied(const wp<IBinder>& who) = 0; };
Back to the implementation of Camera::connect(), new TCam(cameraId) generates a BnCameraClient object, which is defined in the ICameraClient.h file and inherited from the template class BnInterface. The getCameraService() method returns the service proxy BpCameraService of CameraService, and BpCameraService also inherits from the template class BnInterface. Then send the CONNECT command through Binder communication, when BnCameraService receives the CONNECT command, call the connect() member function of CameraService to do corresponding processing.frameworks\av\camera\CameraBase.cpp
template <typename TCam, typename TCamTraits> sp<TCam> CameraBase<TCam, TCamTraits>::connect(int cameraId, const String16& clientPackageName, int clientUid) { ALOGV("%s: connect", __FUNCTION__); sp<TCam> c = new TCam(cameraId); sp<TCamCallbacks> cl = c; const sp<ICameraService>& cs = getCameraService(); if (cs != 0) { c->mCamera = cs->connect(cl, cameraId, clientPackageName, clientUid); } if (c->mCamera != 0) { c->mCamera->asBinder()->linkToDeath(c); c->mStatus = NO_ERROR; } else { c.clear(); } return c; }class BnCameraService: public BnInterface<ICameraService> { public: virtual status_t onTransact( uint32_t code, const Parcel& data, Parcel* reply, uint32_t flags = 0); };
Note: The connect() function is declared as a pure virtual function in ICameraService, the parent class of BpCameraService and BnCameraService, and the implementations are given in BpCameraService and CameraService respectively. BpCameraService is used as a proxy class to provide an interface to the client, and it is actually implemented in the subclass of BnCameraService In CameraService.
In BpCameraService, the connect() function is implemented as follows:
frameworks\av\camera\ ICameraService.cpp
// connect to camera service virtual sp<ICamera> connect(const sp<ICameraClient>& cameraClient, int cameraId, const String16 &clientPackageName, int clientUid) { Parcel data, reply; data.writeInterfaceToken(ICameraService::getInterfaceDescriptor()); data.writeStrongBinder(cameraClient->asBinder()); data.writeInt32(cameraId); data.writeString16(clientPackageName); data.writeInt32(clientUid); remote()->transact(BnCameraService::CONNECT, data, &reply); return interface_cast<ICamera>(reply.readStrongBinder()); }
First, convert the passed Camera object cameraClient into IBinder type, write the call parameters into Parcel (which can be understood as a Binder communication pipeline), send a message through BpBinder's transact() function, and then BnCameraService responds to the connection, and finally Just wait for the server to return, and if successful, generate a BpCamera instance.
真正的服务端响应实现在BnCameraService的onTransact()函数中，其负责解包收到的Parcel并执行client端的请求的方法。
status_t BnCameraService::onTransact( uint32_t code, const Parcel& data, Parcel* reply, uint32_t flags) { switch(code) { case GET_NUMBER_OF_CAMERAS: { CHECK_INTERFACE(ICameraService, data, reply); reply->writeInt32(getNumberOfCameras()); return NO_ERROR; } break; case GET_CAMERA_INFO: { CHECK_INTERFACE(ICameraService, data, reply); CameraInfo cameraInfo; memset(&cameraInfo, 0, sizeof(cameraInfo)); status_t result = getCameraInfo(data.readInt32(), &cameraInfo); reply->writeInt32(cameraInfo.facing); reply->writeInt32(cameraInfo.orientation); reply->writeInt32(result); return NO_ERROR; } break; case CONNECT: { CHECK_INTERFACE(ICameraService, data, reply); sp<ICameraClient> cameraClient = interface_cast<ICameraClient>(data.readStrongBinder()); int32_t cameraId = data.readInt32(); const String16 clientName = data.readString16(); int32_t clientUid = data.readInt32(); sp<ICamera> camera = connect(cameraClient, cameraId, clientName, clientUid); reply->writeStrongBinder(camera->asBinder()); return NO_ERROR; } break; case CONNECT_PRO: { CHECK_INTERFACE(ICameraService, data, reply); sp<IProCameraCallbacks> cameraClient = interface_cast<IProCameraCallbacks>(data.readStrongBinder()); int32_t cameraId = data.readInt32(); const String16 clientName = data.readString16(); int32_t clientUid = data.readInt32(); sp<IProCameraUser> camera = connect(cameraClient, cameraId, clientName, clientUid); reply->writeStrongBinder(camera->asBinder()); return NO_ERROR; } break; case ADD_LISTENER: { CHECK_INTERFACE(ICameraService, data, reply); sp<ICameraServiceListener> listener = interface_cast<ICameraServiceListener>(data.readStrongBinder()); reply->writeInt32(addListener(listener)); return NO_ERROR; } break; case REMOVE_LISTENER: { CHECK_INTERFACE(ICameraService, data, reply); sp<ICameraServiceListener> listener = interface_cast<ICameraServiceListener>(data.readStrongBinder()); reply->writeInt32(removeListener(listener)); return NO_ERROR; } break; default: return BBinder::onTransact(code, data, reply, flags); } }
主要的处理包括：
- 通过data中Camera的Binder对象生成Camera客户代理BpCameraClient实例；
- 将生成的BpCameraClient对象作为参数传递到CameraService（/frameworks/av/services/camera /libcameraservice/CameraService.cpp）的connect()函数中，该函数会返回一个BpCamera实例；
- 将在上述实例对象以IBinder的形式打包到Parcel中返回。
status_t CameraClient::initialize(camera_module_t *module) { int callingPid = getCallingPid(); status_t res; LOG1("CameraClient::initialize E (pid %d, id %d)", callingPid, mCameraId); // Verify ops permissions res = startCameraOps(); if (res != OK) { return res; } char camera_device_name[10]; snprintf(camera_device_name, sizeof(camera_device_name), "%d", mCameraId); mHardware = new CameraHardwareInterface(camera_device_name); res = mHardware->initialize(&module->common); if (res != OK) { ALOGE("%s: Camera %d: unable to initialize device: %s (%d)", __FUNCTION__, mCameraId, strerror(-res), res); mHardware.clear(); return NO_INIT; } mHardware->setCallbacks(notifyCallback, dataCallback, dataCallbackTimestamp, (void *)mCameraId); // Enable zoom, error, focus, and metadata messages by default enableMsgType(CAMERA_MSG_ERROR | CAMERA_MSG_ZOOM | CAMERA_MSG_FOCUS | CAMERA_MSG_PREVIEW_METADATA | CAMERA_MSG_FOCUS_MOVE); LOG1("CameraClient::initialize X (pid %d, id %d)", callingPid, mCameraId); return OK; }
上述函数中，主要注意以下流程：
- 加粗的代码CameraHardwareInterface新建了了一个Camera硬件接口，当然，camera_device_name为摄像头设备名；
- mHardware->initialize(&module->common)调用底层硬件的初始化方法；
- mHardware->setCallbacks将CamerService处的回调函数注册到HAL处。
CameraHardwareInterface定义了Camera的硬件抽象特征，由此进入到HAL。
四、HAL：CameraHardwareInterface
CameraHardwareInterface的作用在于链接Camera Server和V4L2，通过实现CameraHardwareInterface可以屏蔽不同的driver对Camera Server的影响。CameraHardwareInterface同样虚拟继承自RefBase。
CameraHardwareInterface includes a control channel and a data channel. The control channel is used to handle functions such as start/stop of preview and video acquisition, taking photos, and auto-focus. The data channel obtains data such as preview, video recording, and auto-focus through callback functions. When it is necessary to support new hardware, it needs to inherit from CameraHardwareInterface to realize the corresponding functions. The public methods provided by CameraHardwareInterface are as follows:class CameraHardwareInterface : public virtual RefBase { public: CameraHardwareInterface(const char *name) { mDevice = 0; mName = name; } …… }
In the previous section, the initialize() function called mHardware->initialize and mHardware -> setCallbacks, let's look at the implementation of CameraHardwareInterface.h.
status_t initialize(hw_module_t *module)
{
ALOGI("Opening camera %s", mName.string());
int rc = module->methods->open(module, mName.string(),
(hw_device_t **)&mDevice);
if (rc != OK) {
ALOGE("Could not open camera %s: %d", mName.string(), rc);
return rc;
}
initHalPreviewWindow();
return rc;
}In the initialize() method, open the camera module through module->methods->open, initHalPreviewWindow() is used to initialize the related stream opspreview_stream_ops of Preview, and initialize the preview window of hal.
Hardware abstraction layer:
module->methods->open is directly called to hardware\imx\mx6\libcamera\CameraModule.cpp
static struct hw_module_methods_t camera_module_methods = {
open: camera_device_open
};
camera_module_t HAL_MODULE_INFO_SYM = {
common: {
tag: HARDWARE_MODULE_TAG,
version_major: 1,
version_minor: 0,
id: CAMERA_HARDWARE_MODULE_ID,
name: "Freescale CameraHal Module",
author: "Freescale",
methods: &camera_module_methods,
dso: NULL, /* remove compilation warnings */
reserved: { 0 }, /* remove compilation warnings */
},
get_number_of_cameras: camera_get_number_of_cameras,
get_camera_info: camera_get_camera_info,
};
/* open device handle to one of the cameras
*
* assume camera service will keep singleton of each camera
* so this function will always only be called once per camera instance
*/
int camera_device_open(const hw_module_t *module,
const char *name,
hw_device_t **device)
{
int rv = 0;
int num_cameras = 0;
int cameraid;
fsl_camera_device_t *camera_device = NULL;
camera_device_ops_t *camera_ops = NULL;
CameraHal *camera = NULL;
char *SelectedCameraName;
android::Mutex::Autolock lock(gCameraHalDeviceLock);
ALOGI("camera_device open: %s", name);
if (name != NULL) {
cameraid = atoi(name);
num_cameras = camera_get_number_of_cameras();
if (cameraid > num_cameras)
{
ALOGE("camera service provided cameraid out of bounds, "
"cameraid = %d, num supported = %d",
cameraid, num_cameras);
rv = -EINVAL;
goto fail;
}
camera_device = (fsl_camera_device_t *)malloc(sizeof(*camera_device));
if (!camera_device)
{
ALOGE("camera_device allocation fail");
rv = -ENOMEM;
goto fail;
}
camera_ops = (camera_device_ops_t *)malloc(sizeof(*camera_ops));
if (!camera_ops)
{
ALOGE("camera_ops allocation fail");
rv = -ENOMEM;
goto fail;
}
memset(camera_device, 0, sizeof(*camera_device));
memset(camera_ops, 0, sizeof(*camera_ops));
camera_device->base.common.tag = HARDWARE_DEVICE_TAG;
camera_device->base.common.version = 0;
camera_device->base.common.module = (hw_module_t *)(module);
camera_device->base.common.close = camera_device_close;
camera_device->base.ops = camera_ops;
camera_ops->set_preview_window = camera_set_preview_window;
camera_ops->set_callbacks = camera_set_callbacks;
camera_ops->enable_msg_type = camera_enable_msg_type;
camera_ops->disable_msg_type = camera_disable_msg_type;
camera_ops->msg_type_enabled = camera_msg_type_enabled;
camera_ops->start_preview = camera_start_preview;
camera_ops->stop_preview = camera_stop_preview;
camera_ops->preview_enabled = camera_preview_enabled;
camera_ops->store_meta_data_in_buffers =
camera_store_meta_data_in_buffers;
camera_ops->start_recording = camera_start_recording;
camera_ops->stop_recording = camera_stop_recording;
camera_ops->recording_enabled = camera_recording_enabled;
camera_ops->release_recording_frame = camera_release_recording_frame;
camera_ops->auto_focus = camera_auto_focus;
camera_ops->cancel_auto_focus = camera_cancel_auto_focus;
camera_ops->take_picture = camera_take_picture;
camera_ops->cancel_picture = camera_cancel_picture;
camera_ops->set_parameters = camera_set_parameters;
camera_ops->get_parameters = camera_get_parameters;
camera_ops->put_parameters = camera_put_parameters;
camera_ops->send_command = camera_send_command;
camera_ops->release = camera_release;
camera_ops->dump = camera_dump;
*device = &camera_device->base.common;
camera_device->cameraid = cameraid;
camera = new CameraHal(cameraid);
if (!camera)
{
ALOGE("Couldn't create instance of CameraHal class");
rv = -ENOMEM;
goto fail;
}
if (camera->initialize(sCameraInfo[cameraid]) < 0) {
rv = -EINVAL;
goto fail;
}
gCameraHals[cameraid] = camera;
gCamerasOpen++;
}
return rv;
fail:
if (camera_device) {
free(camera_device);
camera_device = NULL;
}
if (camera_ops) {
free(camera_ops);
camera_ops = NULL;
}
if (camera) {
delete camera;
camera = NULL;
}
*device = NULL;
return rv;
}hardware\imx\mx6\libcamera\CameraHal.cpp
status_t CameraHal::initialize(const CameraInfo& info)
{
status_t ret = NO_ERROR;
FLOG_RUNTIME("initialize name:%s, path:%s", info.name, info.devPath);
mDeviceAdapter = DeviceAdapter::Create(info);
if (mDeviceAdapter == NULL) {
FLOGE("CameraHal: DeviceAdapter create failed");
return BAD_VALUE;
}
ret = mDeviceAdapter->initialize(info);
if (ret) {
FLOGE("CameraHal: DeviceAdapter initialize failed");
return ret;
}
mCameraBridge = new CameraBridge();
if (mCameraBridge == NULL) {
FLOGE("CameraHal: new CameraBridge failed");
return BAD_VALUE;
}
ret = mCameraBridge->initialize();
if (ret) {
FLOGE("CameraHal: CameraBridge initialize failed");
return ret;
}
mCameraBridge->getSupportedRecordingFormat(mSupportedRecordingFormat,
MAX_VPU_SUPPORT_FORMAT);
mCameraBridge->getSupportedPictureFormat(mSupportedPictureFormat,
MAX_PICTURE_SUPPORT_FORMAT);
ret = mDeviceAdapter->initParameters(mParameters,
mSupportedRecordingFormat,
MAX_VPU_SUPPORT_FORMAT,
mSupportedPictureFormat,
MAX_PICTURE_SUPPORT_FORMAT);
if (ret) {
FLOGE("CameraHal: DeviceAdapter initParameters failed");
return ret;
}
ret = mCameraBridge->initParameters(mParameters);
if (ret) {
FLOGE("CameraHal: CameraBridge initParameters failed");
return ret;
}
mDeviceAdapter->setErrorListener(mCameraBridge.get());
mCameraBridge->setCameraFrameProvider(mDeviceAdapter.get());
mCameraBridge->setCameraEventProvider(CameraEvent::EVENT_INVALID,
mDeviceAdapter.get());
mBufferProvider = NULL;
return ret;
}
Reference document: http://www.cnblogs.com/younghao/p/5337058.html