Viewing software is an open source image viewing software on the openKylin operating system, which supports basic operations on pictures, such as zooming, flipping, viewing details, copying, printing, renaming, etc., and can also crop, store, and Annotation and ocr (text recognition).
Figure 1: Viewing software interface
As an image viewing software, viewing pictures is its basic function and the most important function. In version V1.2.0 of the picture viewing software, 10 image formats (exr, psd, jfi, jif, jng, wbmp, xbm, xpm, jp2, j2k) are added for viewing and saving. These formats are technically Implemented through the FreeImage library. The following will focus on introducing the image codec library used in the image viewing software—FreeImage.
1. Introduction to picture codec library of viewing software
The image viewing software in the openKylin system currently supports a total of 30 image formats, namely: bmp, jpeg, jpg, jpe, pnm, pgm, ppm, pbm, sr, ras, dib, png, apng, gif, webp, tga, svg, ico, tiff, tif, exr, psd, jfi, jif, jng, wbmp, xbm, xpm, jp2, j2k.
In order to support the above image formats, the image viewing software uses the following libraries for image encoding and decoding: opencv library, FreeImage library, apng library, gif library , etc. Among them, pictures in half formats use the opencv library codec that everyone is familiar with, and pictures in individual formats, such as svg, have their own related libraries. In addition, the FreeImage library is used to read and write pictures. During the use, we found that the FreeImage library is fast, convenient and easy to use for the upper application.
2. Introduction to FreeImage library
FreeImage library is an open source, free and cross-platform image codec library. Supports the processing of more than 20 popular graphics formats, such as BMP, JPEG, GIF, PNG, TIFF, etc.
When using the FreeImage library, libfreeimage-dev and libfreeimageplus-dev must be installed. At the same time, the FreeImage library provides many interfaces for obtaining bitmap information, which are fast, flexible, and easy to use. All the functions in the FreeImage library start with FreeImage_. For example, the read and write functions of image files are FreeImage_Load and FreeImage_Save, and the conversion between them and opencv is also very simple. Interested friends can go to the FreeImage official website for more details .
3. Use the FreeImage library to load pictures
The whole process of loading or manipulating a picture in the picture viewing software is to store the picture in the cv::mat matrix. When opening a picture, the complete process of using the FreeImage library to load the picture in the picture viewing software is as follows:
1. Get the real format of the picture;
2. Determine whether the image supports FreeImage reading;
3. FreeImage loads pictures and obtains FIBITMAP;
4. Convert FIBITMAP to cv::mat;
5. Delete the pictures loaded by libfreeimage from memory to prevent memory leaks.
3.1 Get the real format of the picture
When operating pictures, the suffix of the picture may be .xbm, .sr, etc., but this does not mean that the picture is in xbm or sr format. In this case, you need to use the library function to obtain the real format of the picture.
// Use the library to get the file format, path is the path of the image. QByteArray temp_path; temp_path.append(path.toUtf8()); FREE_IMAGE_FORMAT format = FreeImage_GetFileType(temp_path.data());
FreeImage_GetFileType: Get the file type from the file header. Parameters: image path.
The return value of this function is FREE_IMAGE_FORMAT. As you can see from the figure below, the return value may also be FIF_UNKNOWMN.
Figure 2 Image Type
If the file format parsed from the library function is FIF_UNKNOWMN, we will analyze the image format again by judging the file header from the perspective of file data to increase the success rate of getting the correct file format.
QFile file(path);
if (!file.open(QIODevice::ReadOnly)) { return FIF_UNKNOWN;
}
const QByteArray data = file.read(64);
/* Check bmp file */
if (data.startsWith("BM")) { s return FIF_BMP;
}
//path is the image path
3.2 Determine whether to support reading
After getting the file type and before loading the picture, we need to do another job: determine whether the format can be read by the FreeImage library. Among them, FreeImage_FIFSupportsReading is used to judge whether to support the read operation of the bitmap type.
3.3 Loading pictures
Assuming we have got the image in the correct format, and the format can be read by the FreeImage library. Then call the library function FreeImage_Load to load the bitmap, and the return value is FIBITMAP. The FIBITMAP data structure stores bitmap information and pixel data, and is the core of FreeImage.
3.4 Convert FIBITMAP to mat
In the picture viewing software, the data of the entire process of reading and writing pictures is transmitted in cv::mat matrix. The reason why cv::mat is used is to be able to expand the existing functions of the image viewing software in the future, especially the AI direction provided by opencv.
Figure 3 The software structure for viewing pictures
Therefore, after getting the FIBITMAP returned by FreeImage, we need to convert it to cv::mat.
When converting FIBITMAP to cv::mat, the first thing to do is to see what parameters are needed to construct a mat matrix of a picture.
Mat(int rows, int cols, int type, void* data, size_t step=AUTO_STEP);//number of rows rows, number of columns cols, type type, pointer data to user data, number of bytes occupied by each row.
So, our next task is to get all the parameters we need from FIBITMAP.
1. int rows, int cols
For the number of rows and columns, FreeImage has functions that can be called directly to get the rows and columns.
FIBITMAP *back; int width = FreeImage_GetWidth(dib); int height = FreeImage_GetHeight(dib)；
2. int type
For type, you need to do a little bit of processing. The FreeImage library provides methods to view image depth and data type.
int bpp = FreeImage_GetBPP(src);//The depth of the picture FREE_IMAGE_TYPE fit = FreeImage_GetImageType(src);//Return the data type of the bitmap
After getting the enumeration value of the type of FreeImage, it can be converted to the data type of cv::mat one by one.
Figure 4 data type
3. void* data
Pointer data to user data.
There are also functions in the library that can be called directly.
FreeImage_GetBits(FIBITMAP *dib): Returns a pointer to the data bits of the bitmap
4. size_t step
step The number of bytes occupied by each line
FreeImage_GetPitch(FIBITMAP *dib): Returns bit depth or line width (also called scan width). Returns the width of the bitmap aligned to the next 32-bit byte boundary in bytes.
FIBITMAP *back; int step = FreeImage_GetPitch(dib)；
3.5 memory release
FIBITMAP *back; FreeImage_Unload(dib);//Delete the loaded image from memory to prevent memory leaks
In addition to the FreeImage library, there are currently many excellent image codec libraries. The openKylin image viewing software will adapt more image libraries to support more formats, and use the characteristics of opencv to expand some special functions. Dear friends, please look forward to it!
The openKylin (Open Kylin) community aims to take "co-creation" as the core, on the basis of open source, voluntariness, equality, and collaboration, to build a partner ecosystem with enterprises in an open source and open way, and to jointly create a top-level desktop operating system community. Promote the prosperity and development of Linux open source technology and its software and hardware ecology.
The first batch of council member units in the community include Kylin Software, Puhua Basic Software, Zhongke Fangde, Kylin Principal, Meditation Software, Yiming Software, ZTE New Fulcrum, Yuanxin Technology, China Electronics 32, Jide System, Beijing Lin Zhuo, Advanced Operating System Innovation Center and other 13 industry colleagues and industry organizations.
Source: openKylin
Review: openKylin