ISP (Image Signal Processor), i.e., an image signal processor for processing an image signal output from the image sensor signal. It occupies a dominant position in the core camera system, is an important equipment constitute camera.
The main internal constitution
As shown below, comprising an internal ISP CPU, SUP IP, IF and other equipment, in fact, may be an ISP that the SOC, various algorithms can run programs, real time image signals. 
-
CPU
That is the central processor CPU can run AF, LSC, and other image processing algorithms, control peripheral devices. Modern inside ISP CPU generally ARM Cortex-A series, e.g. Cortex-A5, Cortex-A7.
-
SUB IP
SUB IP is a generic term of various functional modules, each specialized for image processing. Common SUB IP such as DIS, CSC, VRA and so on.
-
Image transmission interface
Image transmission interface is mainly divided into two, parallel and serial ITU CSI. CSI is the MIPI CSI short, in view of the many advantages of the MIPI CSI, cameras in mobile phones have been widely used MIPI-CSI transmission of image data and various interfaces custom data. Generally comprises an external ISP MIPI-CSIS and MIPI-CSIM two interfaces. Built-in ISP generally only need to MIPI-CSIS interface.
-
General Peripherals
General peripheral device means I2C, SPI, PWM, UART, WATCHDOG like. ISP contains I2C controller for reading the OTP information, controls the VCM and the like. For external ISP, ISP itself or I2C slave device. AP may be controlled by the operating mode ISP I2C, acquiring its operating status.
Main features
ISP image processing device as the core, has a very important function, the diagram shows the basic flow of ISP processing image data.
Here are the main features for the next ISP's conduct.
-
DEMOSAIC
DEMOSAIC is one of the main functions of the ISP. SENSOR pixel coverage on the CFA, the CFA is irradiated by light to the pixel. CFA by the R, G, B three colors composition hood, each allowing only a color through a hood, the signal output from each pixel comprises only one color of the three R, G, B of information. Such data is output BAYER SENSOR data, commonly referred to as RAW data. Obvious, RAW data reflect the color information is not true color information. DEMOSAIC is through interpolation algorithm will be true color of each pixel represents calculated.
-
FOCUS
The optical knowledge in the on focal plane on the sensor when the subject is most clear image. LENS by changing the position of the scene so that a clear image on the sensor, the desired function is ISP FOCUS tasks. FOCUS divided into manual and automatic modes. ISP can run CONTRAST AF, PDAF, LASER AF autofocus algorithms.
-
EXPOSURE
exposure. The main effects of light and shade image of EXPOSURE. ISP need to achieve AE function, by controlling the level of exposure, so that the appropriate image brightness.
-
WB
White balance. White balance associated with the color temperature, color is used to measure the authenticity and accuracy of the image. ISP need to implement the AWB function, aimed at a variety of complex scenarios can be accurately restore the original color of the object.
-
LSC
And to eliminate inconsistencies surrounding the image center of the image, including both luminance and chrominance. ISP calibration data required by LSC function OTP is completed.
-
GAMMA CORRECTION
Gamma correction. In response to the light sensor and the response of the human eye to light it is different. Gamma correction is to make the image appear in line with the characteristics of the human eye.
-
CROP/RESIZE
Trimming, i.e., changing the size of the image. Images with different resolutions may be used for output.
-
ASK
Visual Identity. For identifying a specific scene, such as face recognition, license plate recognition. VRA ISP through a variety of algorithms to accurately identify the particular scene.
-
DRC
Dynamic range correction. I.e., the dynamic range of brightness of the image segment. DRC can make the dark scene will not underexposed and bright at the scene would not overexposed. ISP needs to support DRC function.
-
CSC
Color space conversion. For example, ISP will RGB signal into a YUV signal.
-
IS
Image stabilization. IS main role is to make an image not because of a slight shake when hand-held and blurred. There are many IS, e.g. OIS, DIS, EIS. ISP can be achieved DIS and EIS.
In fact, ISP addition to the main functions mentioned above, also need to support DENOISE, CONTRAST, SATURATION, SHARPNESS adjustment and other functions.
control method
Talking about here is the control of the ISP AP control methods.
-
I2C / SPI
This is typically an external ISP practices. Usually for downloading firmware SPI, I2C control registers are generally used. Driving the core ISP, the ISP is typically implemented as an external I2C device, and then packaged into V4L2-SUBDEV.
-
MEM MAP
This is usually a built-ISP approach. The ISP's internal register address space mapped into the kernel address space,
-
MEM SHARE
It is also built-ISP approach. AP allocates memory side, and then pass the memory address to the ISP, are in fact share the same memory. Hence AP will operate on this shared memory real-time feedback to the ISP end.
ISP Infrastructure Solutions
He mentioned several times above the external ISP and built-in ISP, which is actually the ISP's infrastructure program.
External ISP architecture
External ISP architecture refers AP separately arranged outside the image signal processing chip ISP. FIG external ISP architecture generally as follows:
The advantages of an external ISP architecture are:
-
Be able to provide better image quality
In the fierce market competition, we can survive up to now external ISP manufacturers in this area generally have deep knowledge, accumulated a wealth of image quality debugging experience to provide better than the built-in ISP performance and results. Therefore, high-quality external ISP can provide professional and excellent image quality.
-
It can support a richer design and planning
Selection of basic unaffected by external ISP AP, so Meizu selection of the most suitable devices can be from all the excellent ISP chip supplier of many products in order to design more outstanding products.
-
Product differentiation can be achieved
Built-ISP is encapsulated inside the AP, and AP is closely linked, if the AP the same, then the ISP is the same. Therefore, based on the same AP produced mobile phones, their ISP's performance is the same, for tuning condition is fixed, so as not conducive to achieving product differentiation. And if you choose an external ISP, then with an AP, it can be used with different types of ISP, so you can achieve product differentiation, to provide users with a richer and quality products.
Shortcoming external ISP architecture are:
-
High cost price
External ISP needs to be purchased separately, the price is often a lot of money, but also some special features also need to pay an additional fee. Using an external ISP, the need for additional schematic design and LAYOUT, require the use of additional components.
-
Long development cycle
External ISP-driven design requires a little extra effort and time. When using an external ISP, ISP AP drive suppliers can not be used, ISP need to write additional external drive design. In addition, in order to be a perfect match and AP, will maximize the effect, but also often need to pay more debugging effort. Also mentioned above, the use of an external ISP, the need for additional schematic design and LAYOUT, require the use of additional components, which also takes time for processing.
Built-in ISP architecture
ISP architecture built inside an AP means embedded ISP IP, AP directly inside the ISP performs image signal processing. FIG built ISP architecture generally as follows:
Advantage of the built-in ISP architecture are:
-
Reduce the cost price
Built-in ISP embedded within the AP, as there is no need to purchase additional external ISP as required, and does not occupy PCB space, without the need for a separate external circuit design, so you can save BOM, and reduce costs. Given that most users in an important position in the purchase of mobile phones will be the price factor, thus reducing the cost can effectively reduce the end-product prices, it is conducive to dominate the market.
-
Can improve the speed to market products
AP ISP and built in close connection, without the need for schematic design and design LAYOUT, it is possible to reduce development cycles, accelerate product speed to market.
-
Can reduce development effort
If you use the built-in ISP, then the AP vendors can provide relevant information early, drive developers can have enough time to be familiar with the relevant information, and there will be no problem adapting software version, there was no platform architecture compatibility issues. However, if you use an external ISP, then the ISP vendors often do not provide a platform for code / data, and there are generally compatible version of the software problem, which requires the developer to pay the amount of driving experience and time.
Of course there are built using the corresponding ISP deficiencies, particularly see the above analysis, not repeat them here.
In fact, given the importance of the ISP, in order to promote its AP, AP enhance its competitiveness, the built-in ISP AP now more and more powerful, its performance is sufficient to meet the needs of the mobile phone market. Coupled with its series of advantages, more and more mobile phones using the built-in ISP program.
Epilogue
ISP is an important part of a camera system, camera system to understand, can not fail to be familiar with ISP. From the ISP internal structure, features, and other control methods perspective on the knowledge of all aspects of ISP. In addition, the article also detailed analysis of the advantages and disadvantages of both architectures ISP program, hoping to be helpful.
