Article directory
Welcome to follow the WeChat public account Wuxian
Preface
不积跬步，无以至千里；不积小流，无以成江海。
This chapter will serve as the beginning of the Android Camera series, and relevant articles will continue to be output in the future to form a complete series of content as much as possible. It may take a long time to complete all updates, because there are actually a lot of things about Camera, and I need to continue to learn and deepen them. Since I don’t know much about the content of the Camera driver at present, the driver content will be discussed at the end. I also hope that everyone will understand and wait patiently. If there are errors in the article or if you have anything you want to communicate, you can communicate with me directly. we study together!
We will first introduce the startup process of Camera-related services, and then explain the Camera business calling process. The reason is that the Camera service will perform a series of initializations when the system is turned on, so that upper-layer applications can save some time and resources when using the Camera function, thus speeding up the efficiency of the entire Camera business process. We will first talk about the startup process of the Camera HAL service, and then explain the startup process of the cameraserver service. Finally, we will analyze the specific implementation of each Camera business (such as preview, photo taking, video recording, etc.) in detail from the calling process of the upper-layer application.
So this chapter starts with the startup process of Camera HAL service. I wanted to use one article to introduce the startup process of Camera HAL. As a result, the more I wrote, the more code I wrote. It will not be easy to be coherent, which will affect the reading experience, so it is divided into It is introduced in multiple chapters, and I hope it can be explained thoroughly, coherently and clearly.
Android Camera overall architecture
The overall architecture of Camera conforms to the standard Android architecture, that is, App->FWK->HAL->Kernel. However, before Android 8.0, HAL's so library was dynamically loaded by the service of the System partition. For camera hal, it was The cameraserver process is loaded dynamically, and the hal and cameraserver processes belong to the same process. After Android 8.0, Google added the Treble mechanism, placing HAL in an independent process and located in the vendor partition. The System service accesses the HAL process of the vendor partition through binder. For the camera, the cameraserver process accesses the Camera HAL process through binder, which is started through init when booting. In this way, the manufacturer's private implementation can be isolated from the aosp released by Google. If the vendor's vendor partition is not modified during system upgrade, the vendor partition does not need to be upgraded, but the vendor must implement HAL according to the standard interface defined by Google.
CameraProvider
Camera HAL runs in the CameraProvider process. It has reached version 2.7, but the most used version is 2.4. 2.4 is the first version of the CameraProvider process after adding the Treble mechanism, so we start here.
The CameraProvider process is a native service, started through init.rc
( the real process name is [email protected]_64, here we call it the CameraProvider process for abbreviation )
// hardware/interfaces/camera/provider/2.4/default/[email protected]_64.rc
service vendor.camera-provider-2-4 /vendor/bin/hw/android.hardware.camera.provider@2.4-service_64
interface android.hardware.camera.provider@2.4::ICameraProvider legacy/0
class hal
user cameraserver
group audio camera input drmrpc
ioprio rt 4
capabilities SYS_NICE
task_profiles CameraServiceCapacity MaxPerformance
As can be seen from the above, the binary /vendor/bin/hw/[email protected]_64 is executed when the process starts. Next, we find the entry file and entry function of this binary from Android.bp.
// hardware/interfaces/camera/provider/2.4/default/Android.bp
...
cc_binary {
name: "[email protected]_64",
defaults: ["camera_service_defaults"],
compile_multilib: "64",
init_rc: ["[email protected]_64.rc"],
}
cc_defaults {
name: "camera_service_defaults",
defaults: ["hidl_defaults"],
proprietary: true,
relative_install_path: "hw",
srcs: ["service.cpp"], // 二进制入口肯定时service.cpp里面的main函数
shared_libs: [
"[email protected]",
"[email protected]",
"[email protected]",
"[email protected]",
"[email protected]",
"[email protected]",
"[email protected]",
"[email protected]",
"[email protected]",
"[email protected]",
"[email protected]",
"[email protected]",
"libbinder",
"libcamera_metadata",
"libcutils",
"libhardware",
"libhidlbase",
"liblog",
"libutils",
],
static_libs: [
"[email protected]",
],
header_libs: [
"[email protected]_headers",
"[email protected]_headers",
"[email protected]_headers",
"[email protected]_headers",
],
}
...
As can be seen from the above Android.bp file, /vendor/bin/hw/[email protected]_64 only contains one source file service.cpp, so the entry function must be the main function in this cpp. .
// hardware/interfaces/camera/provider/2.4/default/service.cpp
int main()
{
ALOGI("[email protected] legacy service is starting.");
// The camera HAL may communicate to other vendor components via
// /dev/vndbinder
// 这里表示Camera hal可以跟其他 vendor hal进程通过binder通信，否则不可以
android::ProcessState::initWithDriver("/dev/vndbinder");
// b/166675194
if (property_get_bool("ro.vendor.camera.provider24.disable_mem_init", false)) {
if (mallopt(M_BIONIC_ZERO_INIT, 0) == 0) {
// Note - heap initialization is only present on devices with Scudo.
// Devices with jemalloc don't have heap-init, and thus the mallopt
// will fail. On these devices, you probably just want to remove the
// property.
ALOGE("Disabling heap initialization failed.");
}
}
status_t status;
if (kLazyService) {
status = defaultLazyPassthroughServiceImplementation<ICameraProvider>("legacy/0",
/*maxThreads*/ 6);
} else {
// kLazyService就是会延迟初始化，我们这里不细讲，后面单出一个章节介绍一下
// 启动时会执行下面的函数，这里记下这里的泛型类型为ICameraProvider，还有两个参数，后面有用
status = defaultPassthroughServiceImplementation<ICameraProvider>("legacy/0",
/*maxThreads*/ 6);
}
return status;
}
The implementation of defaultPassthroughServiceImplementation is as follows:
// system/libhidl/transport/include/hidl/LegacySupport.h
// 这里的两个泛型参数都为ICameraProvider
template <class Interface, class ExpectInterface = Interface>
__attribute__((warn_unused_result)) status_t defaultPassthroughServiceImplementation(
const std::string& name, size_t maxThreads = 1) {
// 设置该进程中binder通信线程池的最大线程数，这里maxThreads=6
configureRpcThreadpool(maxThreads, true);
// 注意这里的泛型类型为ICameraProvider， 参数name为legacy/0
status_t result = registerPassthroughServiceImplementation<Interface, ExpectInterface>(name);
if (result != OK) {
return result;
}
// 把当前线程也就是主线程加入binder线程池
joinRpcThreadpool();
return UNKNOWN_ERROR;
}
registerPassthroughServiceImplementation will eventually execute the following function:
// system/libhidl/transport/include/hidl/LegacySupport.h
template <class Interface, class ExpectInterface = Interface>
__attribute__((warn_unused_result)) status_t registerPassthroughServiceImplementation(
const std::string& name = "default") {
// 这里Interface和ExpectInterface是ICameraProvider
// ICameraProvider::descriptor为[email protected]::ICameraProvider
// 至于ICameraProvider::descriptor的值哪里赋值的，后面细讲
return registerPassthroughServiceImplementation(Interface::descriptor,
ExpectInterface::descriptor, name);
}
// system/libhidl/transport/LegacySupport.cpp
__attribute__((warn_unused_result)) status_t registerPassthroughServiceImplementation(
const std::string& interfaceName, const std::string& expectInterfaceName,
const std::string& serviceName) {
// 这里前两个参数为[email protected]::ICameraProvider，
// 第三个参数为lambda表达式的一个函数，该函数传入两个参数，然后执行registerAsServiceInternal
// 第四个参数为legacy/0
return details::registerPassthroughServiceImplementation(
interfaceName, expectInterfaceName,
[](const sp<IBase>& service, const std::string& name) {
return details::registerAsServiceInternal(service, name);
},
serviceName);
}
// system/libhidl/transport/LegacySupport.cpp
// 这个函数主要干了两件事情
// 1. 打开camera hal的so库，初始化Camera Hal，比如获取camera numbers，在getRawServiceInternal里实现
// 2. 将camera hal注册为binder服务，在registerServiceCb里实现，也就是上一步的lamda函数，最终调用的是
// registerAsServiceInternal
__attribute__((warn_unused_result)) status_t registerPassthroughServiceImplementation(
const std::string& interfaceName, const std::string& expectInterfaceName,
RegisterServiceCb registerServiceCb, const std::string& serviceName) {
// 这是一个关键函数，第一个参数为[email protected]::ICameraProvider
// 第二个参数为 legacy/0
// 我们下面先分析这个函数，然后再继续
sp<IBase> service =
getRawServiceInternal(interfaceName, serviceName, true /*retry*/, true /*getStub*/);
// 这里我们把后面的代码先省略掉，等getRawServiceInternal函数讲完之后再继续往下分析
...
}
getRawServiceInternal
// system/libhidl/transport/ServiceManagement.cpp
// 第一个参数为[email protected]::ICameraProvider
// 第二个参数为 legacy/0
// 后两个参数都为true
sp<::android::hidl::base::V1_0::IBase> getRawServiceInternal(const std::string& descriptor,
const std::string& instance,
bool retry, bool getStub) {
using Transport = IServiceManager1_0::Transport;
sp<Waiter> waiter;
sp<IServiceManager1_1> sm;
Transport transport = Transport::EMPTY;
if (kIsRecovery) {
transport = Transport::PASSTHROUGH;
} else {
sm = defaultServiceManager1_1();
if (sm == nullptr) {
ALOGE("getService: defaultServiceManager() is null");
return nullptr;
}
// 这里最终会去读取/vendor/etc/vintf/manifest.xml文件来获取当前transport类型
// 初始文件在device/xxx/manifest.xml，编译时跟其他manifest
// 一起写入/vendor/etc/vintf/manifest.xml，定义如下：
/*
<manifest version="1.0" type="device" target-level="3">
<hal format="hidl">
<name>android.hardware.camera.provider</name>
<transport>hwbinder</transport>
<fqname>@2.4::ICameraProvider/legacy/0</fqname>
</hal>
</manifest>
*/
// transport类型决定了当前HAL是绑定式还是直通式
Return<Transport> transportRet = sm->getTransport(descriptor, instance);
if (!transportRet.isOk()) {
ALOGE("getService: defaultServiceManager()->getTransport returns %s",
transportRet.description().c_str());
return nullptr;
}
transport = transportRet;
}
const bool vintfHwbinder = (transport == Transport::HWBINDER); // true
const bool vintfPassthru = (transport == Transport::PASSTHROUGH); // false
const bool trebleTestingOverride = isTrebleTestingOverride(); // false
const bool allowLegacy = !kEnforceVintfManifest || (trebleTestingOverride && isDebuggable()); // false
const bool vintfLegacy = (transport == Transport::EMPTY) && allowLegacy; // false
if (!kEnforceVintfManifest) {
ALOGE("getService: Potential race detected. The VINTF manifest is not being enforced. If "
"a HAL server has a delay in starting and it is not in the manifest, it will not be "
"retrieved. Please make sure all HALs on this device are in the VINTF manifest and "
"enable PRODUCT_ENFORCE_VINTF_MANIFEST on this device (this is also enabled by "
"PRODUCT_FULL_TREBLE). PRODUCT_ENFORCE_VINTF_MANIFEST will ensure that no race "
"condition is possible here.");
sleep(1);
}
// 因为参数getStub为true,所以这里的for循环不进入，不执行
for (int tries = 0; !getStub && (vintfHwbinder || vintfLegacy); tries++) {
if (waiter == nullptr && tries > 0) {
waiter = new Waiter(descriptor, instance, sm);
}
if (waiter != nullptr) {
waiter->reset(); // don't reorder this -- see comments on reset()
}
Return<sp<IBase>> ret = sm->get(descriptor, instance);
if (!ret.isOk()) {
ALOGE("getService: defaultServiceManager()->get returns %s for %s/%s.",
ret.description().c_str(), descriptor.c_str(), instance.c_str());
break;
}
sp<IBase> base = ret;
if (base != nullptr) {
Return<bool> canCastRet =
details::canCastInterface(base.get(), descriptor.c_str(), true /* emitError */);
if (canCastRet.isOk() && canCastRet) {
if (waiter != nullptr) {
waiter->done();
}
return base; // still needs to be wrapped by Bp class.
}
if (!handleCastError(canCastRet, descriptor, instance)) break;
}
// In case of legacy or we were not asked to retry, don't.
if (vintfLegacy || !retry) break;
if (waiter != nullptr) {
ALOGI("getService: Trying again for %s/%s...", descriptor.c_str(), instance.c_str());
waiter->wait(true /* timeout */);
}
}
if (waiter != nullptr) {
waiter->done();
}
// getStub为true，进入这里的if语句
if (getStub || vintfPassthru || vintfLegacy) {
// getPassthroughServiceManager 返回new PassthroughServiceManager()
const sp<IServiceManager1_0> pm = getPassthroughServiceManager();
if (pm != nullptr) {
// 下面单独分析PassthroughServiceManager的get函数
sp<IBase> base = pm->get(descriptor, instance).withDefault(nullptr);
// getStub为true， trebleTestingOverride为false，这里的if进不去
if (!getStub || trebleTestingOverride) {
base = wrapPassthrough(base);
}
return base;
}
}
return nullptr;
}
PassthroughServiceManager.get
// system/libhidl/transport/ServiceManagement.cpp
// descriptor： [email protected]::ICameraProvider
// instance: legacy/0
sp<IBase> base = pm->get(descriptor, instance)
Return<sp<IBase>> get(const hidl_string& fqName,
const hidl_string& name) override {
sp<IBase> ret = nullptr;
// 主要就是执行了openLibs函数，我们先把参数理一下，然后在进入openLibs函数进行分析
// 第一个参数fqName: [email protected]::ICameraProvider
// 第二个参数是一个lamda函数,这个函数有三个参数，这里主要是这个lamda函数的实现，
// 这里我们先省略，到调用的时候再详细讲
openLibs(fqName, [&](void* handle, const std::string &lib, const std::string &sym) {
...
}
}
PassthroughServiceManager.openLibs
// system/libhidl/transport/ServiceManagement.cpp
// openLibs函数实现
static void openLibs(
// fqName: [email protected]::ICameraProvider
const std::string& fqName,
// eachLib: 就是上面提到的lamda函数
const std::function<bool /* continue */ (void* /* handle */, const std::string& /* lib */,
const std::string& /* sym */)>& eachLib) {
//fqName looks like [email protected]::IFoo
// 用::符号将参数分割为[email protected]和ICameraProvider
size_t idx = fqName.find("::");
if (idx == std::string::npos ||
idx + strlen("::") + 1 >= fqName.size()) {
LOG(ERROR) << "Invalid interface name passthrough lookup: " << fqName;
return;
}
// 这里拿到的就是[email protected]
std::string packageAndVersion = fqName.substr(0, idx);
// 这里拿到的就是 ICameraProvider
std::string ifaceName = fqName.substr(idx + strlen("::"));
// prefix为[email protected]
const std::string prefix = packageAndVersion + "-impl";
// sym为 HIDL_FETCH_ICameraProvider
const std::string sym = "HIDL_FETCH_" + ifaceName;
// 到此，上面参数已经解析完毕，下面根据prefix和sym来执行
constexpr int dlMode = RTLD_LAZY;
void* handle = nullptr;
dlerror(); // clear
static std::string halLibPathVndkSp = details::getVndkSpHwPath();
std::vector<std::string> paths = {
HAL_LIBRARY_PATH_ODM, HAL_LIBRARY_PATH_VENDOR, halLibPathVndkSp,
#ifndef __ANDROID_VNDK__
HAL_LIBRARY_PATH_SYSTEM,
#endif
};
// 这里翻译下上面的paths变量：
std::vector<std::string> paths = {
"/odm/lib64/hw/", "/vendor/lib64/hw/", "/apex/com.android.vndk.v33/lib64/hw/",
#ifndef __ANDROID_VNDK__
"/system/lib64/hw/",
#endif
};
// 条件为false，不进入
if (details::isTrebleTestingOverride()) {
// Load HAL implementations that are statically linked
handle = dlopen(nullptr, dlMode);
if (handle == nullptr) {
const char* error = dlerror();
LOG(ERROR) << "Failed to dlopen self: "
<< (error == nullptr ? "unknown error" : error);
} else if (!eachLib(handle, "SELF", sym)) {
return;
}
}
// 在paths的几个路径中寻找[email protected]文件
// 我们到设备里查看一下，发现在/vendor/lib64/hw目录下，因为是64位系统，所以/vendor/lib/hw/是不会遍历的
// 130|emulator_car_x86_64:/ # find . -name [email protected]
// ./vendor/lib/hw/[email protected]
// ./vendor/lib64/hw/[email protected]
for (const std::string& path : paths) {
std::vector<std::string> libs = findFiles(path, prefix, ".so");
for (const std::string &lib : libs) {
// fullPath=/vendor/lib64/hw/[email protected]
const std::string fullPath = path + lib;
if (kIsRecovery || path == HAL_LIBRARY_PATH_SYSTEM) {
handle = dlopen(fullPath.c_str(), dlMode);
} else {
#if !defined(__ANDROID_RECOVERY__) && defined(__ANDROID__)
// 最终执行这里，load so库，拿到handle
handle = android_load_sphal_library(fullPath.c_str(), dlMode);
#endif
}
if (handle == nullptr) {
const char* error = dlerror();
LOG(ERROR) << "Failed to dlopen " << lib << ": "
<< (error == nullptr ? "unknown error" : error);
continue;
}
// 还记得eachLib函数吗，就是上面省略掉的lamda函数，下面开始分析
if (!eachLib(handle, lib, sym)) {
return;
}
}
}
}
Let’s start to explain the implementation of eachLib lamda
// fqName： [email protected]::ICameraProvider
// name: legacy/0
Return<sp<IBase>> get(const hidl_string& fqName,
const hidl_string& name) override {
sp<IBase> ret = nullptr;
// 这里的handle为dlopen /vendor/lib64/hw/[email protected]后的handle
// lib参数为[email protected]
// sym参数为HIDL_FETCH_ICameraProvider
openLibs(fqName, [&](void* handle, const std::string &lib, const std::string &sym) {
IBase* (*generator)(const char* name);
// 通过dlsym在[email protected]中找到HIDL_FETCH_ICameraProvider的地址
*(void **)(&generator) = dlsym(handle, sym.c_str());
if(!generator) {
const char* error = dlerror();
LOG(ERROR) << "Passthrough lookup opened " << lib << " but could not find symbol "
<< sym << ": " << (error == nullptr ? "unknown error" : error)
<< ". Keeping library open.";
// dlclose too problematic in multi-threaded environment
// dlclose(handle);
return true; // continue
}
// 执行HIDL_FETCH_ICameraProvider函数，参数为legacy/0
ret = (*generator)(name.c_str());
if (ret == nullptr) {
LOG(ERROR) << "Could not find instance '" << name.c_str() << "' in library " << lib
<< ". Keeping library open.";
// dlclose too problematic in multi-threaded environment
// dlclose(handle);
// this module doesn't provide this particular instance
return true; // continue
}
// Actual fqname might be a subclass.
// This assumption is tested in vts_treble_vintf_test
using ::android::hardware::details::getDescriptor;
std::string actualFqName = getDescriptor(ret.get());
CHECK(actualFqName.size() > 0);
registerReference(actualFqName, name);
return false;
});
return ret;
}
HIDL_FETCH_ICameraProvider
// hardware/interfaces/camera/provider/2.4/default/CameraProvider_2_4.cpp
// 参数name为legacy/0
ICameraProvider* HIDL_FETCH_ICameraProvider(const char* name) {
using namespace android::hardware::camera::provider::V2_4::implementation;
ICameraProvider* provider = nullptr;
if (strcmp(name, kLegacyProviderName) == 0) {
// 执行这里
provider = getProviderImpl<LegacyCameraProviderImpl_2_4>();
} else if (strcmp(name, kExternalProviderName) == 0) {
provider = getProviderImpl<ExternalCameraProviderImpl_2_4>();
} else {
ALOGE("%s: unknown instance name: %s", __FUNCTION__, name);
}
return provider;
}
// 创建CameraProvider对象并返回，在CameraProvider的构造函数中，会执行IMPL的构造函数，
// 也就是LegacyCameraProviderImpl_2_4的构造函数
template<typename IMPL>
CameraProvider<IMPL>* getProviderImpl() {
CameraProvider<IMPL> *provider = new CameraProvider<IMPL>();
if (provider == nullptr) {
ALOGE("%s: cannot allocate camera provider!", __FUNCTION__);
return nullptr;
}
if (provider->isInitFailed()) {
ALOGE("%s: camera provider init failed!", __FUNCTION__);
delete provider;
return nullptr;
}
return provider;
}
Constructor implementation of LegacyCameraProviderImpl_2_4
// hardware/interfaces/camera/provider/2.4/default/LegacyCameraProviderImpl_2_4.cpp
LegacyCameraProviderImpl_2_4::LegacyCameraProviderImpl_2_4() :
camera_module_callbacks_t({
sCameraDeviceStatusChange,
sTorchModeStatusChange}) {
mInitFailed = initialize();
}
The LegacyCameraProviderImpl_2_4 object was created above and its initialization function initialize() was executed.
To summarize, it actually loads the [email protected] library when the CameraProvider process starts and executes the initialization function of LegacyCameraProviderImpl_2_4.
Okay, this section will be introduced here first, and the explanation will continue in the next section.