In ubuntu, cross-compile harfbuzz, freetype, opencv (including WITH_QT), and transplant it to the linux development board

1. Introduction

1.1 Brief description

1. WITH_QT is an auxiliary for opencv. If you only want to use opencv to process images without using a window to display images, you do not need to check WITH_QT.

2. Freetype is also an auxiliary to opencv. Freetype mainly enables opencv to support Chinese characters. If you want to use putText() to add Chinese characters to pictures, you need to compile freetype and harfbuzz. If you don't have this requirement, you don't need to cross-compile freetype and harfbuzz.

3. WITH_QT can also enable opencv to support Chinese display, that is, opencv uses the font library in QT without relying on freetype. However, when opencv uses the function addText() provided by QT to add Chinese characters to the picture, a window must be created first. This is set in the QT source code and cannot be modified after cross-compilation. (Perhaps you can comment out the cvAddText() function in qt before cross-compiling if (!guiMainThread) CV_Error(CV_StsNullPtr, "NULL guiReceiver (please create a window)");)

4. For the Chinese display of opencv, you can also rewrite opencv's putText() yourself to expand opencv from supporting ascii codes to supporting unicode characters.
   To add Chinese character support to opencv, please refer to these two articles: 1. Use OpenCV to add Chinese annotations to images 2. Opencv renders Chinese characters

   The freetype method mentioned in link 1 and link 2 means that after cross-compiling freetype, you have to write the relevant calling code yourself. But in fact, opencv has officially provided a method to call freetype ( https://github.com/opencv/opencv_contrib/tree/4.x/modules/freetype ), so for subsequent tests, I directly used the opencv official example.

1.2 opencv version and its background

1. Cross-compilation tool chain: gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabihf
2. ubuntu platform：ubuntu18.04.1 64bit
3. opencv and opencv_contrib versions: opencv-4.5.5 and opencv_contrib-4.5.5
4. Embedded linux development board kernel: NXP CORTEX-A7 IMX6ULL
5. cmake version: cmake-3.25.0 (select the new version for the compilation tool, with fewer bugs)

1.3 Steps

1. Cross-compile freetype, harfbuzz, and some of their related dependent libraries
2. Cross compile Qt
3. Cross compile opencv
4. Ported to linux development board
5. test

2. Cross-compile freetype, harfbuzz and related dependent libraries

2.1 Compilation steps

1. Cross-compile zlib library (libpng, opencv dependent library)
2. Cross-compile libpng library (freetype dependent library)
3. Cross-compile bzip2 library (freetype extension library, not required)
4. Cross compile the icu library (harfbuzz depends on the library). hafbuzz cross compile depends on at least one of Builtin, Glib and ICU. I choose the ICU library here.
5. Cross-compile harfbuzz and freetype (freetype and harfbuzz depend on each other)
6. Cross-compile eigen, Eigen is an open source library based on C++ templates that supports linear algebra, matrix and vector operations, numerical analysis and related algorithms. (Not required)
   Here is a Baidu network disk link. You can download the relevant libraries from the official website, or you can get them directly from here:
   Link: https://pan.baidu.com/s/1haqsW4uPdyIV_C2Pi-WRaA
   Extraction code: cm8m

2.2 Cross-compile zlib

1. Download zlib, http://zlib.net/ . The version I am using here is 1.2.11.
2. Unzip: tar -xzvf zlib-1.2.11.tar.gz
3. cd zlib-1.2.11
4. Set the compiler: export CC=arm-linux-gnueabihf-gcc (because there is no option to configure the compiler in zlib's configure, so set the environment variable here)
5. Configure the compilation environment: ./configure --prefix=$PWD/tmp --enable-shared
6. make
7. make install
8. cp -rfp /home/book/opencvarm/zlib-1.2.11/tmp/* /home/book/arm-opencv ( I put all the libraries in the arm-opecv directory to facilitate packaging to the arm development board later )

2.3 Cross-compile libpng

1. Download libpng, http://www.libpng.org/pub/png/libpng.html . The version I am using here is 1.6.37
2. Unzip: tar xJf libpng-1.6.37.tar.xz
3. cd libpng-1.6.37
4. Configure the compilation environment: ./configure --host=arm-linux-gnueabihf --prefix=$PWD/tmp --build=x86_64-linux-gnu -with-zlib-prefix=/home/book/arm-opencv
5. If you use prefix= to specify the output directory during the compilation and installation of zlib, an error that the zlib library cannot be found will be displayed when the libpng library is installed. Follow the following steps: ① cp scripts/makefile.linux makefile ② vi
   makefile
   ③
   Find zlib related Modify the two lines into the actual zlib/lib and zlib/include, then find the prefix item and modify it to the directory you want to prefix.
   If it is cross-compiled here, you also need to modify gcc on the cc side to your system. Cross-compiler, such as my cross-compiler: arm-linux-gnueabihf-gcc
   ④ Save and make, make install
6. cp -rfp /home/book/opencvarm/libpng-1.6.37/tmp/* /home/book/arm-opencv

2.4 Cross-compile bzip2

1. Download bzip2, https://sourceware.org/bzip2/downloads.html . The version I am using here is 1.0.6
2. Unzip: tar xvzf bzip2-1.0.6.tar.gz
3. cd bzip2-1.0.6
4. bzip2 does not provide configure, so you need to modify the Makefile-libbz2_so to generate a dynamic library and modify the makefile.
5. vi Makefile-libbz2_so，修改CC=arm-linux-gnueabihf-gcc SHELL=/bin/bash
6. make -f Makefile-libbz2_so
7. Without configure, modify the Makefile directly:
   SHELL=/bin/bash
   CC=arm-linux-gnueabihf-gcc
   AR=arm-linux-gnueabihf-ar
   RANLIB=arm-linux-gnueabihf-ranlib
   PREFIX=/home/book/opencvarm/bzip2 -1.0.6/tmp
   Finally delete the test after all: libbz2.a bzip2 bzip2recover (bzip2 will generate an executable file of bzip2 after compilation. Since we use cross tools for compilation, this executable file is for the ARM system. , naturally it cannot be executed on the host machine, which will cause the test to fail.)
8. make
9. make install
10. cp -rfp /home/book/opencvarm/bzip2-1.0.6/tmp/* /home/book/arm-opencv
11. cp /home/book/opencvarm/bzip2-1.0.6/{libbz2.so.1.0.6,libbz2.so.1.0} /home/book/arm-opencv/lib
12. Note: Here you can use the command readelf -h ./bzip2to view the executable file system of bzip2, or readelf -h ./libbz2.aview the library file system, or directly use the file command, such as file ./bzip2. Check if the file is for ARM architecture.

2.5 Cross-compile ICU

1. Note: I first compiled icu56.1 and icu65.1 according to the official tutorial, but neither was successful. Finally, attempts to compile icu69 and icu72.1 were successful (the compilation steps did not change). Finally, the reason is summarized: Checking the icu official website, I found that icu officially classifies all icu69 and below as old versions. The old versions may not be maintained, and the compilation fails anyway. Therefore, when choosing a library version, you should first check the stable version of the library and the most commonly used version. If not, choose the latest version to try. After all, the version is updated and the new version must have solved some of the bugs that existed before .
2. Download ICU, https://icu.unicode.org/download , here I choose the latest version icu72.1.
3. tar xvzf icu-release-72-1.tar.gz
4. cd ./icu-release-72-1/icu4c/source
5. mkdir arm-build
6. cd arm-build
7. …/runConfigureICU Linux/gcc
8. make
9. cd …
10. ./configure CXXFLAGS=-std=c++11 --host=arm-linux-gnueabihf --build=x86_64-linux-gnu --target=arm-linux-gnueabihf --prefix=$PWD/tmp --with-cross-build=/home/book/opencvarm/icu-release-72-1/icu4c/source/arm-build --disable-samples --disable-tests CC=arm-linux-gnueabihf-gcc CXX=arm-linux-gnueabihf-g++ AR=arm-linux-gnueabihf-ar
11. make
12. make install
13. cp -rfp /home/book/opencvarm/icu-release-72-1/icu4c/source/tmp/* /home/book/arm-opencv

2.6 Cross-compile harfbuzz and freetype

1. Because harfbuzz and freetype depend on each other, you must either compile harfbuzz (without freetype) first, then compile freetype (with harfbuzz), and then compile harfbuzz (with freetype); or vice versa, compile freetype first. I choose to compile harfbuzz first .
2. Download harfbuzz, https://github.com/harfbuzz/harfbuzz , I choose version 2.6.4 here. Download freetype, https://freetype.org/download.html , I choose version 2.10.2 here.
3. 交叉编译harfbuzz（不带freetype）
   ① 解压：tar xJf harfbuzz-2.6.4.tar.xz
   ② cd harfbuzz-2.6.4
   ③ ./configure --host=arm-linux-gnueabihf CC=arm-linux-gnueabihf-gcc --prefix=$PWD/tmp --with-freetype=no --with-icu=yes --with-glib=no CPPFLAGS=-I/home/book/arm-opencv/include LDFLAGS=-L/home/book/arm-opencv/lib
   ⑤ make
   ⑥ make install
   ⑦ cp -rfp /home/book/opencvarm/harfbuzz-2.6.4/tmp/* /home/book/arm-opencv
4. 交叉编译freetype（带harfbuzz）
   ① 解压：tar xJf freetype-2.10.2.tar.xz
   ② cd freetype-2.10.2
   ③ ./configure CC=arm-linux-gnueabihf-gcc --host=arm-linux-gnueabihf ZLIB_CFLAGS=-I/home/book/arm-opencv/include ZLIB_LIBS=-L/home/book/arm-opencv/lib BZIP2_CFLAGS=-I/home/book/arm-opencv/include BZIP2_LIBS=-L/home/book/arm-opencv/lib LIBPNG_CFLAGS=-I/home/book/arm-opencv/include LIBPNG_LIBS=-L/home/book/arm-opencv/lib HARFBUZZ_CFLAGS=-I/home/book/arm-opencv/include/harfbuzz HARFBUZZ_LIBS=-L/home/book/arm-opencv/include/lib --prefix=$PWD/tmp
   ④ make
   ⑤ make install
   ⑥ cp -rfp /home/book/opencvarm/freetype-2.10.2/tmp/* /home/book/arm-opencv
5. 再交叉编译harfubuzz（带freetype）
   ① 返回上面harfbuzz源码路径
   ② ./configure --host=arm-linux-gnueabihf CC=arm-linux-gnueabihf-gcc --prefix=$PWD/tmp --with-freetype=yes --with-icu=yes --with-glib=no CPPFLAGS=-I/home/book/arm-opencv/include LDFLAGS=-L/home/book/arm-opencv/lib FREETYPE_CFLAGS=-I/home/book/arm-opencv/include/freetype2 FREETYPE_LIBS=“-L/home/book/arm-opencv/lib -lfreetype -lpng -lbz2 -lz”
   ③ make
   ④ make install
   ⑤ cp -rfp /home/book/opencvarm/harfbuzz-2.6.4/tmp/* /home/book/arm-opencv

2.7 Cross-compile eigen

1. Download eigen, https://eigen.tuxfamily.org/index.php?title=Main_Page , I use the latest version eigen-3.4.0 here.
2. 解去：tar xvzf eigen-3.4.0.tar.gz
3. cd eigen-3.4.0
4. mkdir build (create the directory where eigen is built); mkdir tmp (create the directory where eigen is installed)
5. Compile using cmake:
   
   
   
6. After the build is completed, enter the build directory, make, then make install
7. cp -rfp /home/book/opencvarm/eigen-3.4.0/tmp/* /home/book/arm-opencv

3. Cross-compile Qt

To cross-compile Qt, I refer to the Punctual Atom document "[Punctual Atom] I.MX6U Porting Qt5.12.9_V1.1.pdf", which is very detailed and highly recommended. Because the Zhengdian Atomic document is originally open source, let’s write a path here. Punctual Atomic Download Center----Jundian Atomic Linux Development Board-----i.MX6ULL Linux Alpha Development Board----Data Disk Development Board Data Link (This is a Baidu Cloud Network Disk link)----After entering Just download 10. User Manual.zip------after decompressing, find 08 [Punctual Atom] I.MX6U transplanted Qt5.12.9_V1.1.pdf.

4. Cross-compile opencv

1. Download opencv4.5.5 and opencv_contrib-4.5.5, https://opencv.org/releases/ . The latest version of opencv4 is chosen here because opencv has been updated. CMakeLists.txt must be more and more perfect. It must have fixed many bugs in the previous version and there will be fewer compilation errors .
2. Unzip: unzip opencv-4.5.5.zip, unzip opencv_contrib-4.5.5.zip
3. cd opencv-4.5.5，mkdir build，mkdir tmp
4. Build using cmake-gui:
   
   
5. In the Search column, search WITH_FREETYPR and check it. (If you don’t need Chinese, you can leave it unchecked)
6. In the Search column, search WITH_QT and check it. (If there is no window requirement, you can leave it unchecked). The purpose of WITH_QT is: after opencv is cross-compiled and transplanted to the ARM board, opencv needs to use some window classes (for example, when imshow needs to call the window), but the cross-compiled opencv does not have window class support, opencv can be called in the x86 environment Ubuntu's native window, so WITH_QT's purpose is to provide window class support for opencv on ARM boards. WITH_GTK is a GTK window class. I read many blogs saying that transplanting GTK classes is very complicated, and most of them use QT window classes.
7. Modify CMAKE_BUILD_TYPE to Release.
8. Find OPENCV_GENERATE_PKGCONFIG and check it. This is the option to generate a .pc file. The pc file saves the header file of the library and the path information of the library.
9. 勾选上BUILD_TIFF, BUILD_JPEG, BUILD_PNG, WITH_OPENGL, WITH_EIGEN
10. Check BUILD_ZLIB and add the path to the library: Select the path at ZLIB_LIBRARY_RELEASE: /home/book/arm-opencv/lib/libz.so
11. Add the opencv extension library: Select the path at OPENCV_EXTRA_MODULES_PATH: /home/book/opencv_contrib-4.5.5/modules
12. Select the opencv installation directory: Select the path in CMAKE_INSTALL_PREFIX: /home/book/opencv-4.5.5/tmp
13. ENABLE_NEON. When performing some calculations such as image processing on an ARM chip, the calculation frame rate is often low due to the large amount of calculations. Therefore, it is necessary to choose a simpler and faster calculation method to obtain an improvement in processing speed. ARM NEON is a good choice.
    
14. Add -lpthread -ldl -lrt after CMAKE_EXE_LINKER_FLAGS
15. Cancel WITH_1394. Because OpenCV does not support 1394 interface cameras for camera acquisition, it only supports USB interface cameras.
16. Cancel WITH_WEBP. (optional). WebP is a new image format that provides lossy and lossless compression for images on the Internet. Webmasters and web developers can use the WebP image format to create smaller and richer images to make your web faster. I have no need for this here, so I cancel.
17. After setting the above, click configure.
18. After configure is completed, cancel BUILD_opencv_xfeatures2d, BUILD_opencv_wechar_qrcode, BUILD_opencv_gapi, BUILD_opencv_face. I don't use these modules. If you need to use them and the download fails, such as: In
    
    this case, you can find the corresponding files, such as /home/book/opencvarm/opencv-4.5 in the picture above. 5/arm-build/CMakeDownloadLog.txt, this file records the URL of the required download file. You can download it manually and put it in the corresponding directory. For details, please refer to the blog: https://blog.csdn.net/qq153471503/article/details/123542363
19. There is also an error in the compile box:
    
    Translation: Compatibility with CMake<2.8.12 will be removed from future versions of CMake . The solution is: update the min value of the VERSION parameter or use the ...max suffix to ensure that the project does not need to be compatible with older versions .
    I use the second method here:
    ① cd /home/book/opencvarm/opencv-4.5.5/3rdparty/carotene/hal, then vi CMakeLists.txt, change cmake_minimum_required(VERSION 2.8.8 FATAL_ERROR) to cmake_minimum_required(VERSION 2.8 .8…3.25.10 FATAL_ERROR).
    ② cd /home/book/opencvarm/opencv-4.5.5/3rdparty/carotene/, then vi CMakeLists.txt, change cmake_minimum_required(VERSION 2.8.11 FATAL_ERROR) to cmake_minimum_required(VERSION 2.8.11…3.25.10 FATAL_ERROR).
20. Display freetype2: no; harfbuzz: no in the edit box . It means that the cross-compiled freetype and harfbuzz libraries are not detected.
    ① vi /home/book/opencv_contrib-4.5.5/modules/freetype/CMakeLists.txt to view the construction of freetype:

set(the_description "FreeType module. It enables to draw strings with outlines and mono-bitmaps/gray-bitmaps.")
if(APPLE_FRAMEWORK)
ocv_module_disable(freetype)
endif()
ocv_check_modules(FREETYPE freetype2)
ocv_check_modules(HARFBUZZ harfbuzz)

if(OPENCV_INITIAL_PASS)
  if(NOT FREETYPE_FOUND)
    message(STATUS "freetype2:   NO")
  else()
    message(STATUS "freetype2:   YES (ver ${FREETYPE_VERSION})")
  endif()

  if(NOT HARFBUZZ_FOUND)
    message(STATUS "harfbuzz:    NO")
  else()
    message(STATUS "harfbuzz:    YES (ver ${HARFBUZZ_VERSION})")
  endif()
endif()

if(FREETYPE_FOUND AND HARFBUZZ_FOUND)
  ocv_define_module(freetype opencv_core opencv_imgproc WRAP python)
  ocv_target_link_libraries(${the_module} ${FREETYPE_LIBRARIES} ${HARFBUZZ_LIBRARIES})
  ocv_include_directories( ${FREETYPE_INCLUDE_DIRS} ${HARFBUZZ_INCLUDE_DIRS} )
else()
  ocv_module_disable(freetype)
endif()

From this point of view, the code is executed to message (STATUS "harfbuzz: NO"), that is, freetype and harfbuzz are not detected. So let's rephrase that.
② Modify if(NOT FREETYPE_FOUND) to if(FREETYPE_FOUND)
③ Modify if(NOT HARFBUZZ_FOUND) to if(HARFBUZZ_FOUND)
④ Modify if(FREETYPE_FOUND AND HARFBUZZ_FOUND) to if((NOT FREETYPE_FOUND) AND (NOT HARFBUZZ_FOUND))
⑤ And in c make- Click Add Entry in the gui and add FREETYPE_LIBRARIES, HARFBUZZ_LIBRARIES, FREETYPE_INCLUDE_DIRS, HARFBUZZ_INCLUDE_DIRS. That is, the library path and header file directory of freetype and harfbuzz.

21. Search for qt in the Search bar and select the path in Qt5_DIR: /usr/lib/arm-qt/lib/cmake/Qt5

22. Click Configure and build again

23. Add QT_QMAKE_EXECUTABLE path:/usr/lib/arm-qt/bin/qmake

24. Search for eigen in the Search bar and select the path in Eigen3_DIR: /home/book/arm-opencv/share/eigen3/cmake

25. It is recommended that you finally browse the display box under the Search bar and focus on checking whether the file path is set correctly. Because sometimes there are multiple versions of library files in Ubuntu, you must choose to cross-compile to the arm architecture library instead of the x86_64 architecture library.

26. Click Configure. After the build is successful, click Generate.
    

27. Enter the opencv build directory cd /home/book/opencv-4.5.5/build, and then execute make.
    When executing make, a "file not found error" may occur, such as:
    
    Solution : Copy the lib and include directories of all previously cross-compiled libraries to the lib and include directories of the cross-compilation tool chain.
    For example, my tool chain include location/usr/local/arm/gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabihf/bin/…/lib/gcc/arm-linux-gnueabihf/4.9.4/include and lib location/usr/local/arm/gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabihf/bin/…/lib/gcc/arm-linux-gnueabihf/4.9.4/…/…/…/… /arm-linux-gnueabihf/lib.
    (You can check the include and lib locations through the command echo 'main(){}'| arm-linux-gnueabihf-gcc -E -v)

    ① cd /home/book/arm-opencv
    ② sudo cp include/* -rf /usr/local/arm/gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabihf/bin/…/lib/gcc/arm -linux-gnueabihf/4.9.4/include
    ③ sudo cp lib/* -rfd /usr/local/arm/gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabihf/bin/…/lib/gcc/arm -linux-gnueabihf/4.9.4/…/…/…/…/arm-linux-gnueabihf/lib
    ④ cd /usr/local/arm/gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabihf/bin /…/lib/gcc/arm-linux-gnueabihf/4.9.4/include/freetype2, use the command mv ./* …/ to move the freetype folder to the upper level.
    ⑤ Compile again: make

28. Compilation is complete, make install

29. cp -rfp /home/book/opencvarm/opencv-4.5.5/tmp/* /home/book/arm-opencv

5. Transplant to linux development board

1. Packaging: tar cvf arm-opencv.tar arm-opencv
2. Transfer arm-python.tar to the /home/book directory of the development board through tools such as FileZilla
3. Unzip: tar xvf arm-opencv.tar

6. Test

1. Write test code: vi 111.cpp

#include "opencv2/opencv.hpp"
#include "opencv2/freetype.hpp"
#include "opencv2/core.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/videoio.hpp"

using namespace std;
using namespace cv;

int main(int argc,char **argv)
{
    
    
	String text = "Hello, 加油！冲冲冲#***#";
	int fontHeight = 60;
	int thickness = -1;
	int linestyle = 8;
	int baseline=0;

	cout << "Built with OpenCV " << CV_VERSION << endl;

	/*参数1：窗口的名字.  参数2：窗口类型，CV_WINDOW_AUTOSIZE 时表明窗口大小等于图片大小。不可以被拖动改变大小。
	CV_WINDOW_NORMAL 时，表明窗口可以被随意拖动改变大小。*/
    namedWindow("test", WINDOW_AUTOSIZE);

	Ptr<freetype::FreeType2> ft2;
	ft2 = freetype::createFreeType2();
	//下面的字库要自己下载并拷贝到需要的位置
	ft2->loadFontData( "simsun.ttc", 0 );

	Mat image;
    image = imread("test.jpg");

	Size textSize = ft2->getTextSize(text,
	                             fontHeight,
	                             thickness,
	                             &baseline);
	if(thickness > 0){
    
    
	    baseline += thickness;
	}

	Point textOrg((image.cols - textSize.width) / 2,
	              (image.rows + textSize.height) / 2);
	rectangle(image, textOrg + Point(0, baseline),
	      textOrg + Point(textSize.width, -textSize.height),
	      Scalar(0,255,0),1,8);
	line(image, textOrg + Point(0, thickness),
	 textOrg + Point(textSize.width, thickness),
	 Scalar(0, 0, 255),1,8);
	ft2->putText(image, text, textOrg, fontHeight,
	         Scalar::all(255), thickness, linestyle, true );
	
	imwrite("test.jpg",image);
	
	imshow("test",image);//在“test”这个窗口输出图片。
    waitKey(5000);//等待5秒，程序自动退出。改为0，不自动退出。
	
	return 0;

}

2. 交叉编译111.cpp：arm-linux-gnueabihf-g++ -o 111 111.cpp -std=c++11 -I/home/book/arm-opencv/include/opencv4 -L/home/book/arm-opencv/lib -lopencv_imgcodecs -lopencv_imgproc -lopencv_core -ldl -lm -lpthread -lrt -lopencv_freetype -L/home/book/arm-opencv/lib -lfreetype -lharfbuzz -lpng -lz -lopencv_highgui -L/usr/lib/arm-qt/lib -lQt5Widgets -lQt5Test -lQt5Gui -lQt5Core
3. Get the 111 executable file and transfer the 111 executable file to the development board. First chmod +x 111 to add executable operations to 111; then export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/book/arm-opencv/lib to set environment variables; finally execute ./111. ( Note: the font library and pictures must be in the same directory as the executable file 111 )
4. Effect:
   

7. Reference document links

1. Use OpenCV to add Chinese annotations to images
2. opencv renders Chinese characters
3. Summary of usage of namedWindow() function in opencv (02)
4. opencv cv::addText if no window exception
5. Add freetype support to OPENCV and display Chinese characters (compile opencv and contrib libraries on mac
6. Cross compile freetype2, harfbuzz
7. OpenCV porting and cross-compilation detailed process and problem solving records
8. Introduction to cmake_minimum_required of Cmake command