EVS（External View System)）
Google's introduction: https://source.android.com/devices/automotive/camera-hal?hl=zh-cn#ievscamera
1. Advantages of EVS
EVS personal understanding is to quickly transmit the image data acquired by the camera to the display screen. Compared with traditional cameras, it has the following advantages:
1. Quick start
The design of EVS HAL minimizes the process dependence from camera to display. Once the kernel is started, the display will also be started, and the transmission time between the two is greatly reduced;
2. Scalability
EVS provides a safe way for the application to obtain the feedback of the car camera in a read-only way, so that the application
The terminal can realize some advanced functions such as face recognition, sign detection, traffic report, etc., see config.json for details.
3. Configurability
The HAL layer design of EVS is simpler than the complete camera HAL layer, and on the client side, through config.json to
The camera is configured, and google also provides some demos, you can check them at will
CameraName_Backup, LaneView, right turn camera.
2. The framework of EVS
It uses hal's technology to divide this process into the following levels:
application: application layer, implemented by developers
Evs manager: provide relevant interface services to the application layer
hardware_service: Provide specific interface implementation to the manager
driver: divided into camera driver and display driver
Briefly describe the process: the APP communicates with the hardware service through EVSmanager to obtain the authority and agent to operate the camera and display
The objects are IEvsCamera and IEvsDisplay respectively. Through the proxy object, the interaction between camera and display is realized, and the camera
The image is displayed on the display in time.
3. Detailed process
Let's sort out the examples provided by Android
1. EvsManager and EvsHardware_service initialization
Code path: /packages/services/Car/evs/manager
const static char kHardwareEnumeratorName[] = "EvsEnumeratorHw";
const static char kMockEnumeratorName[] = "EvsEnumeratorHw-Mock";
The first is to register the service when starting up, and then obtain it through getservice(). The registration code is as follows (Note: the code is only a key process, not all codes)
int main(int argc, char** argv) {
ALOGI("EVS manager starting\n");
const char* evsHardwareServiceName = kHardwareEnumeratorName;
std::thread registrationThread(startService, evsHardwareServiceName, kManagedEnumeratorName); //在这里注册startService这个线程
joinRpcThreadpool();
ALOGE("EVS Hardware Enumerator is shutting down");
return 1;
}static void startService(const char *hardwareServiceName, const char * managerServiceName) {
android::sp<Enumerator> service = new Enumerator();
//先初始化hardwareService，在初始化中获取hardservice的服务
if (!service->init(hardwareServiceName)) {
ALOGE("Failed to connect to hardware service - quitting from registrationThread");
exit(1);
}
ALOGI("EVS managed service is starting as %s", managerServiceName);
status_t status = service->registerAsService(managerServiceName);//注册服务
ALOGD("Registration complete");
}bool Enumerator::init(const char* hardwareServiceName) {
mHwEnumerator = IEvsEnumerator::getService(hardwareServiceName);
bool result = (mHwEnumerator.get() != nullptr);
return result;
}The above HardWare_Service is also started and registered in the system through init.rc when the system is turned on, as follows
hardware/interfaces/automotive/evs/1.0/default/[email protected]
HardWard_Service started by the [email protected] command in init.rc, note
The registration process is the same as the EvsManager service above. After the registration is completed, wait for the manager to call.
The entity of HardWard_Service here is implemented as HardWard's EvsEnumerator object, and in its initialization function, the camera in the system will be added to sCameraList.
EvsEnumerator::EvsEnumerator() {
sCameraList.emplace_back(EvsCamera::kCameraName_Backup);
sCameraList.emplace_back("LaneView");
sCameraList.emplace_back("right turn");
}You can see that there are three cameras by default: rear, left view and right view
At this point, the Hardward Service and the upper-level Evs manager and application are all initialized, and the application can pass through
The proxy object of Evs manager interacts with Hardward Service, obtains the proxy objects of Camera and Display, and uses
for subsequent operations.
2. The process of opencamera
Since there are multiple cameras in the system, it is necessary to specify the cameraId to open the corresponding camera, and the cameraId is obtained through the getcameraID() method
mEvs->getCameraList([this, &config](hidl_vec<CameraDesc> cameraList) {
ALOGI("Camera list callback received %zu cameras",
cameraList.size());
for (auto&& cam: cameraList) {
ALOGD("Found camera %s", cam.cameraId.c_str());
bool cameraConfigFound = false;
for (auto&& info: config.getCameras()) {
if (cam.cameraId == info.cameraId) {
// We found a match!
if (info.function.find("reverse") != std::string::npos) {
mCameraList[State::REVERSE].push_back(info);
}
if (info.function.find("right") != std::string::npos) {
mCameraList[State::RIGHT].push_back(info);
}
if (info.function.find("left") != std::string::npos) {
mCameraList[State::LEFT].push_back(info);
}
if (info.function.find("park") != std::string::npos) {
mCameraList[State::PARKING].push_back(info);
}
cameraConfigFound = true;
break;
}
}The parameter of getCameraList is a function pointer, which is passed to Hardward Service through Evs Manager, and is controlled by
Hardward Service callback, pass cameraList to the above callback function. In this callback function, it will traverse
cameraList, and compare it with config.json, if there is a cameraId corresponding to cameraList in config.json, then add this
camera into mCameraInfo.
The specific implementation of getCameraList in Hardward Service is:
Return<void> EvsEnumerator::getCameraList(getCameraList_cb _hidl_cb) {
const unsigned numCameras = sCameraList.size();
std::vector<CameraDesc> descriptions;
descriptions.reserve(numCameras);
for (const auto& cam : sCameraList) {
descriptions.push_back( cam.desc );
}
hidl_vec<CameraDesc> hidlCameras(descriptions);
ALOGD("reporting %zu cameras available", hidlCameras.size());
_hidl_cb(hidlCameras);//这里callback
return Void();
}In the getCameraList method above, the _hidl_cb function passed in will be called back and the EvsEnumerator will be initialized
sCameraList is passed, then the APP will get all the cameraIds, and then call the opencamera() method of the specified cameraID.
openCamera will first call the openCamera method of Enumerator.cpp in Evs Manager:
Return<sp<IEvsCamera>> Enumerator::openCamera(const hidl_string& cameraId) {
sp<HalCamera> hwCamera;//有封装了一个HalCamera
for (auto &&cam : mCameras) {
bool match = false;
cam->getHwCamera()->getCameraInfo([cameraId, &match](CameraDesc desc) {
if (desc.cameraId == cameraId) {
match = true;
}
}
);
if (match) {
hwCamera = cam;
break;
}
sp<IEvsCamera> device = mHwEnumerator->openCamera(cameraId);
sp<VirtualCamera> clientCamera;
if (hwCamera != nullptr) {
clientCamera = hwCamera->makeVirtualCamera();
}
return clientCamera;
}In the openCamera method of the above code Evs Manager Enumerator, it can be summarized as the following points:
1. Use the getCameraInfo method to verify the cameraId, and check whether there is a camera corresponding to the CameraId;
2. Through the openCamera method of mHwEnumerator, call Hardward service to open the corresponding cameraId
camera;
3. Encapsulate the IevsCamera returned by Hardward service as HalCamera;
4. Call the makeVirtualCamera method of HalCamera to configure the buffers of HardWard service camera
information, and will return the value return.
It can be summarized into two main processes:
1）mHwEnumerator->openCamera
2）hwCamera->makeVirtualCamera()
1) mHwEnumerator->openCamera, call the opencamera() method in the hardware service
Return<sp<IEvsCamera>> EvsEnumerator::openCamera(const hidl_string& cameraId) {
CameraRecord *pRecord = nullptr;
for (auto &&cam : sCameraList) {
if (cam.desc.cameraId == cameraId) {
// Found a match!
pRecord = &cam;
break;
}//先匹配到对应的cameraId
}
sp<EvsCamera> pActiveCamera = pRecord->activeInstance.promote();
if (pActiveCamera != nullptr) {
ALOGW("Killing previous camera because of new caller");
closeCamera(pActiveCamera);
}
pActiveCamera = new EvsCamera(cameraId.c_str());
pRecord->activeInstance = pActiveCamera;
return pActiveCamera;
}In the above code Hardward's openCamera method, it can be summarized as follows:
1. Determine whether the incoming cameraId is valid;
2. Whether the corresponding cameraId has been opened, if it has been opened, then closeCamera;
3. Create an EvsCamera object. In the constructor of EvsCamera, assign values to width, height and format, and return them to the EVS Manager.
2) hwCamera->makeVirtualCamera(), the main function is to apply for an image buffer of a specified size, the specific process is as follows
sp<VirtualCamera> HalCamera::makeVirtualCamera() {
//将 hwCamera 封装为VirtualCamera 对象client
sp<VirtualCamera> client = new VirtualCamera(this);
//通过changeFramesInFlight来设置buffer数量，
if (!changeFramesInFlight(client->getAllowedBuffers())) {
client = nullptr;
return nullptr;
}
mClients.push_back(client);
return client;
}The default value in the client->getAllowedBuffers() code is 1, which is set in /packages/services/Car/evs/manager/VirtualCamera.h
Application process:
bool HalCamera::changeFramesInFlight(int delta) {
// Walk all our clients and count their currently required frames
// 上注释也说的很明白，统计每个client所需的frame数量
unsigned bufferCount = 0;
for (auto&& client : mClients) {
sp<VirtualCamera> virtCam = client.promote();
if (virtCam != nullptr) {
bufferCount += virtCam->getAllowedBuffers();
}
}
bufferCount += delta;
//调用setMaxFramesInFlight()方法设置下去
Return<EvsResult> result = mHwCamera->setMaxFramesInFlight(bufferCount);Enter the EvsCamera of hardwareservice
Return<EvsResult> EvsCamera::setMaxFramesInFlight(uint32_t bufferCount) {
if (setAvailableFrames_Locked(bufferCount)) {
return EvsResult::OK;
} else {
return EvsResult::BUFFER_NOT_AVAILABLE;
}bool EvsCamera::setAvailableFrames_Locked(unsigned bufferCount) {
//根据当前情况，申请所需的buffer
if (mFramesAllowed < bufferCount) {
// An increase is required
unsigned needed = bufferCount - mFramesAllowed;
ALOGI("Allocating %d buffers for camera frames", needed);
//通过increaseAvailableFrames_Locked()方法完成buffer申请
unsigned added = increaseAvailableFrames_Locked(needed);
if (added != needed) {
ALOGE("Rolling back to previous frame queue size");
decreaseAvailableFrames_Locked(added);
return false;
}
} else if (mFramesAllowed > bufferCount) {
// A decrease is required
unsigned framesToRelease = mFramesAllowed - bufferCount;
ALOGI("Returning %d camera frame buffers", framesToRelease);
unsigned released = decreaseAvailableFrames_Locked(framesToRelease);
if (released != framesToRelease) {
ALOGE("Buffer queue shrink failed -- too many buffers currently in use?");
}
}
return true;
}unsigned EvsCamera::increaseAvailableFrames_Locked(unsigned numToAdd) {
GraphicBufferAllocator &alloc(GraphicBufferAllocator::get());
unsigned added = 0;
while (added < numToAdd) {
buffer_handle_t memHandle = nullptr;
//根据width、height、format等信息申请buffer
status_t result = alloc.allocate(mWidth, mHeight, mFormat, 1, mUsage,
&memHandle, &mStride, 0, "EvsCamera");
bool stored = false;
for (auto&& rec : mBuffers) {
if (rec.handle == nullptr) {
rec.handle = memHandle;
rec.inUse = false;
stored = true;
break;
}
}
mFramesAllowed++;//在这里++
added++;//已经申请的个数累加，累加的个数应该与传入的参数numToAdd相等
}
return added;
}The above APP has completed the operation of opencamera.
4. Realization of Renesas
The above example shows the relationship between APP, Evsmanager and hardware. Then, for an OEM manufacturer like Renesas, he
What is to be realized is all the API functions of the hardware service. These API definitions
in /hardware/interfaces/automotive/evsthe .hal file in . In the code /vendor/renesas/hal/evs/, it is the function realization of the above API by Renesas, or look at the process of opencamera().
Return<sp<IEvsCamera>> EvsEnumerator::openCamera(const hidl_string& cameraId) {
CameraRecord *pRecord = findCameraById(cameraId);
if (!pRecord) {
ALOGE("Requested camera %s not found", cameraId.c_str());
return nullptr;
}
sp<EvsCamera> pActiveCamera = pRecord->activeInstance.promote();
if (pActiveCamera != nullptr) {
ALOGW("Killing previous camera because of new caller");
closeCamera(pActiveCamera);
}
//创建EvsCamera，初始化width、height等信息
pActiveCamera = new EvsCamera(cameraId.c_str(), pRecord->dim.width, pRecord->dim.height);
pRecord->activeInstance = pActiveCamera;
if (pActiveCamera == nullptr) {
ALOGE("Failed to allocate new EvsCamera object for %s\n", cameraId.c_str());
}
return pActiveCamera;
}Look at EvsCamera
EvsCamera::EvsCamera(const char *id, uint32_t initWidth, uint32_t initHeight) :
mFramesAllowed(0),
mFramesInUse(0),
mStreamState(STOPPED),
mFd(-1),
mBufType(V4L2_BUF_TYPE_VIDEO_CAPTURE),
mMemType(V4L2_MEMORY_USERPTR) {
ALOGD("EvsCamera instantiated");
mDescription.cameraId = id;
//设置基本属性
mWidth = initWidth;
mHeight = initHeight;
mFormat = BUFFER_FORMAT;
mUsage = BUFFER_USAGE;
mComposer = IComposer::getService();
mDisplayWidth = mComposer->getDisplayWidth();
mDisplayHeight = mComposer->getDisplayHeight();
//进行初始化
if(!initialize(id)) {
ALOGE("Failed to open v4l device %s\n", id);
}
}bool EvsCamera::initialize(const char* deviceName)
{
//先打开设备
if ((mFd = open(deviceName, O_RDWR)) == -1) {
ALOGE("Error while opening device %s: %s", deviceName, strerror(errno));
return false;
}
//填充v4l2
v4l2_format format = {};
format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
format.fmt.pix.pixelformat = CAMERA_FORMAT;
format.fmt.pix.width = mWidth;
format.fmt.pix.height = mHeight;
format.fmt.pix.field = V4L2_FIELD_NONE;
//通过ictol与driver通信
if (ioctl(mFd, VIDIOC_S_FMT, &format) < 0) {
ALOGE("VIDIOC_S_FMT: %s", strerror(errno));
return false;
}//VIDIOC_S_FMT这个在文档中对它的解释是：Set scaling and the data format
// Report the current output format
format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
//VIDIOC_G_FMT表示获取format
if (ioctl(mFd, VIDIOC_G_FMT, &format) == 0) {
ALOGI("Current camera output format: fmt=0x%X, %dx%d, pitch=%d",
format.fmt.pix.pixelformat,
format.fmt.pix.width,
format.fmt.pix.height,
format.fmt.pix.bytesperline);
} else {
ALOGE("VIDIOC_G_FMT: %s", strerror(errno));
return false;
}
return true;
}