Learn cmake from mxnet’s project organization structure

illustrate

This series will introduce how MXNet uses CMake to organize projects.

This series of articles covers most of the CMake commands used in actual development.

The code in this article comes from MXNet .

What is CMake? Why learn CMake?

To figure out CMakewhat it does, we need to understand C++the development process of the entire project writing. Due to my limited ability, I will only give a brief introduction here.
After the code is written, the code needs to be compiled through a compiler. When the project becomes large, it becomes a very troublesome thing to manually enter the compilation command after each compilation. In order to solve the problem, manual input is required for each compilation. The problem of compilation command was born Makefile. The so-called Makefileis to write the compilation command of the entire project Makeifleinto a file named. You only need to enter make every time to compile the entire project. Makefile implements many convenient operations, such as searching all source files at once, so you don’t have to Each time you add a source file, write a g++ filename.cpp -c -o filename.ocommand manually.
Although Makefileyou only need to write a compilation command once, when the project becomes huge and you need to use a third-party library, the Makefilemanagement will be a little troublesome. In addition to this, Makefilethere is a more fatal problem. , it cannot support cross-platform very well, that is, if you need to implement it for use on two different platforms, you Makefileoften need to write two Makefilefiles. In order to solve the problems encountered
above , there is . What it can do, here is a description from the official website:MakefileCMakeCMake

CMake is an open-source, cross-platform family of tools designed to build, test and package software. CMake is used to control the software compilation process using simple platform and compiler independent configuration files, and generate native makefiles and workspaces that can be used in the compiler environment of your choice. The suite of CMake tools were created by Kitware in response to the need for a powerful, cross-platform build environment for open-source projects such as ITK and VTK.

The rough meaning of the above in English CMakeis a cross-platform construction tool used to generate Makefile. This is Cmakethe main purpose.
CMakeIt has now become C++a build tool used by many projects, and its use CMakehas become a general trend. So learning CMakeis particularly important.

Introduction to MXNet

MXNetIt is an open source machine learning framework that supports many different languages ​​and can facilitate users to train and deploy machine learning. Everyone is familiar with using Pythoninstallation MXNetfor machine learning training, and MXNetthe code here provides a dynamic library. Developers of different languages ​​will encapsulate it differently, and they will eventually call the dynamic library. MXNetThe dynamic library here is C++written using , and CMakethe project is built using .

MXNet code organization structure

MXNetThere are many subdirectories under the directory, we only need to pay attention to some of its directories and files:

src directory: stores source code and some header files
include directory: stores header files of externally released libraries
test directory: stores test cases
cmake directory: stores .cmakefiles
3rdparty directory: stores third-party library
CMakeList.txtfiles: CMakeconfiguration files

How to use CMake

To use, CMakewe only need to write CMakeList.txtthe file. After completing the writing of the file, we can pass cmakethe command (you need to ensure that it exists in the execution location. CMakeList.txtIf it does not exist, you only need to enter the path of the file after the command). The command will generate Makefilethe file for us. After generating Makefilethe file After that we just need to execute makethe command to start building the project.

CMakeList.txt in MXNet

Next, let's learn MXNethow to use it CMaketo build. In the above code organization structure, we find that in addition to its own source code, the project also uses third-party libraries, and these third parties are actually built through CMake. Such third-party libraries CMakecan be well managed.

cmake_minimum_required(VERSION 3.13)

This command is used to specify cmakethe minimum version number. When the version number is smaller than the specified version number, 3.13it cmakewill not continue to be executed, but will prompt the user to upgrade cmakethe version.

if() elseif() else() endif()

cmakeThe logical control statements in MXNetare as follows:

if(CMAKE_CROSSCOMPILING)
  set(__CMAKE_CROSSCOMPILING ${CMAKE_CROSSCOMPILING})
  set(__CMAKE_CROSSCOMPILING_OVERRIDE ON)
endif()

if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
  if(CMAKE_CXX_COMPILER_VERSION VERSION_LESS 7.0)
    message(FATAL_ERROR "MXNet 2 requires a C++17 compatible compiler. Please update to GCC version 7 or newer.")
  endif()
elseif(CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
  if(CMAKE_CXX_COMPILER_VERSION VERSION_LESS 6.0)
    message(FATAL_ERROR "MXNet 2 requires a C++17 compatible compiler. Please update to Clang version 6 or newer.")
  endif()
endif()

ifThe second parameter in the statement is the judgment condition, which STREQUALis used to judge whether the two strings are equal and VERSION_LESSwhether the previous version number is smaller than the latter version number.
The first ifstatement above will CMAKE_CROSSCOMPILINGbe ONexecuted when , and the judgment statement here should be used for cross-compilation.

Note: The sum CMakein is expressed by sum .truefalseONOFF

Note: Cross-compilation is mostly used to implement code compilation for embedded devices. Since the performance of embedded devices is limited, it is generally compiled on a host with strong performance. The compiled target file (or library) can be transplanted to the embedded device for use. .

set()

setCommands CMakeare used to define variables in, for example:

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS ON)

If you want to get a certain variable, you can ${VARIABLE_NAME}do it through.
setCommands can also set buffer variables, which can be used globally.

cmake built-in variables

Generally, CMAKEthe variables starting with them are cmakebuilt-in variables. You can obtain the value of each variable according to the method of obtaining the variable value. setThe values ​​of these variables can be changed through commands to achieve some special purposes. MXNetThe built-in variables used in:

• CMAKE_CROSSCOMPILING cross-platform compilation macro, this macro will be activated after specifying the system name
• CMAKE_CXX_STANDARD is used to specify the C++ version
• CMAKE_CXX_STANDARD_REQUIRED When turned on, the version specified in CMAKE_CXX_STANDARD will be used. If not, it will stop and report an error. If not, the previous version will be used when turned off.
• CMAKE_CXX_EXTENSIONS Whether to enable compiler extensions for C++
• CMAKE_CXX_COMPILER_ID The name of the compiler such as GNU
• CMAKE_CXX_COMPILER_VERSION Compiler version number
• CMAKE_PROJECT_NAME The name of the top-level project (if a project contains a sub-project, then CMAKE_ PROJECT_NAME in the sub-project will be the name of the top-level project, not the name of its own project)
• PROJECT_NAME The name of the current project
• CMAKE_CURRENT_SOURCE_DIR The absolute path of the source directory currently being processed

project()

This command is used to specify the name of the project and the programming language used by the project. MXNetThe example in specifying the project name as mxnetthe programming language is C/C++:

project(mxnet C CXX)

message()

Used to print prompt information, similar to log output, which can specify the output level. MXNetThe following are used level:

message(FATAL_ERROR "MXNet 2 requires a C++17 compatible compiler. Please update to GCC version 7 or newer.")
message(STATUS "CMAKE_CROSSCOMPILING ${CMAKE_CROSSCOMPILING}")
message(WARNING "Could not find NVML libraries")
message(ERROR " Only one of USE_JEMALLOC and USE_GPERFTOOLS can be defined at once")
message("After choosing blas, linking to ${mxnet_LINKER_LIBS}")

Note: The log level determines the processing that will occur and which execution FATAL_ERROEwill be stopped . Note: When no level is entered, it means the message is an important one and the user should pay attention. Note: There is no such level in , but this level is used in , and no errors occur. This is something I don't quite understand.cmake

cmakeERRORMXNet

include()

includeCan be used to introduce macros, modules and a file.

• If you introduce a file (usually with the suffix .cmake), the usage at this time is consistent with the macro usage C/C++in includemxnet. The usage of introducing a file in mxnet is as follows:

  include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/Utils.cmake)
  

  The above command will import Utils.cmakesome commonly used tool functions and macros defined in this file. \

• mxnetModules are also includeintroduced through commands in , for example:

  include(CMakeDependentOption)
  

  CmakeDependentOptionThe main methods supported in this module are introduced above cmake_dependent_option.

cmake_dependent_option()

This method is used to set some macro variables. The macro variables are set to one value when the dependency is established, and set to another value when the dependency is not met, such as an example mxnetin:

cmake_dependent_option(USE_NVML "Build with nvml support if found" ON "USE_CUDA" OFF)

USE_NVMLIt will be when USE_CUDAit is done . It will be when it is done . A more complex example:ONONOFFOFF

cmake_dependent_option(A "this is description" ON "B;NOT C" OFF)

The above code means that when Bit is opened or Cclosed, Ait will be opened; in other cases, Ait will be closed.

option()

Used to set an option. The variable of this option can be used to set its value when executing the cmake command -DOPTION_NAME=ON/OFF.

option(USE_NCCL "Use NVidia NCCL with CUDA" OFF)

For example, an option is set above USE_NCCL, and its default value is OFFthe string in the middle as its description.

To be updated, last updated on 2023.6.23

find_package()

execute_process()

string()

add_compile_definitions()

add_definitions()

check_language()

enable_language()

find_program()