xmake v2.2.2, let C/C++ have package dependencies and build automatically

Preface

After more than four months, xmake finally updated the new version v2.2.2, and launched a heavyweight feature: natively supported remote dependency package management.

As for this feature, I actually wrote about it for nearly a year before it was initially completed. For the development progress and history of this feature, interested students can refer to related issues: #69 .

• Project source code
• Official document

The current implementation effects are as follows, with exactly the same semantic version dependency description:

Fully consistent cross-platform build behavior, one-click xmake compilation:

Complete project description:

add_requires("tbox 1.6.*", "libpng ~1.16", "zlib")

target("test")
    set_kind("binary")
    add_files("src/*.c") 
    add_packages("tbox", "libpng", "zlib")

Let me briefly introduce the background of this function:

When we write C/C++ programs, the use of third-party dependent libraries has always been a difficult problem, because the build system of each dependent library is different, and the difference in the support of the code platform makes it impossible to use it like other high-level languages. Convenient and easy-to-use package management support.

Although there are package management tools such as homebrew and vcpkg to solve this problem, there are some limitations, such as:

1. homebrew does not support iphoneos, android, windows platforms
2. vcpkg does not support semantic version selection, multi-version management
3. In addition, it does not support project management and construction

For the current cross-platform build tools, there is a lack of built-in package management support. For example, cmake only provides find_packageto find system packages. Although it can be used in conjunction with third-party package management such as vcpkg, I personally find it not very convenient.
This will cause other users of the project to install vcpkg or install dependent libraries on the system when compiling. It is better for the pc platform. For the dependent libraries of the iphoneos, android and other platforms, users have to toss. After a while.

The idea of ​​xmake is:真正的一致维护, 真正的一键编译

• Consistency of build behavior: Regardless of whether your project has library dependencies or tool dependencies, you only need to execute one xmakecommand to compile and pass.
• Consistency of project maintenance: Regardless of whether your project is used on windows or linux, iphone, android, you only need a xmake.lua maintenance project.

And cmake also needs to generate additional third-party IDE project files. Even if cmakelist.txt is the same, it is impossible for users to guarantee complete consistency in the construction and maintenance experience. After all, they are still limited by tools such as vc/make.

Currently supported features

• Semantic version support, for example: ">= 1.1.0 <1.2", "~1.6", "1.2.x", "1.*"
• Provide multi-warehouse management support such as official package warehouse, self-built private warehouse, project built-in warehouse, etc.
• Cross-platform package compilation and integration support (packages of different platforms and different architectures can be installed at the same time, and quickly switch to use)
• Debug dependent package support to realize source code debugging

Dependent package handling mechanism

Here we briefly introduce the processing mechanism of the entire dependency package:

1. Priority check whether there is a specified package in the current system directory and third-party package management. If there is a matching package, then there is no need to download and install (of course, you can also set not to use the system package)
2. Retrieve the package that matches the corresponding version, then download, compile, and install (Note: Install in a specific xmake directory, which will not interfere with the system library environment)
3. Compile the project, and finally automatically link the enabled dependent packages

Get started quickly

Create an empty project that depends on the tbox library:

$ xmake create -t console_tbox test
$ cd test

Just execute the compilation. If the tbox library is not currently installed, it will be automatically downloaded and installed and used:

$ xmake

Switch to the iphoneos platform for compilation, and the tbox library of the iphoneos version will be reinstalled for linking use:

$ xmake f -p iphoneos
$ xmake

Switch to the android platform arm64-v8a architecture compilation:

$ xmake f -p android [--ndk=~/android-ndk-r16b]
$ xmake

Semantic version setting

The dependency package management of xmake fully supports semantic version selection, for example: "~1.6.1". For a detailed description of the semantic version, please see: http://semver.org/

Some semantic version writing:

add_requires("tbox 1.6.*", "pcre 1.3.x", "libpng ^1.18")
add_requires("libpng ~1.16", "zlib 1.1.2 || >=1.2.11 <1.3.0")

At present, the semantic version parser used by xmake is the sv library contributed by uael . There is also a detailed description of the version description. You can refer to the following: version description

Of course, if we have no special requirements for the version of the current dependency package, we can write it directly like this:

add_requires("tbox", "libpng", "zlib")

This will use the latest known version of the package, or a package compiled from the source code of the master branch. If the current package has a git repo address, we can also specify a specific branch version:

add_requires("tbox master")
add_requires("tbox dev")

Additional package information settings

Optional package settings

If the specified dependent package is not supported by the current platform, or the compilation and installation fails, then xmake will compile an error. This is reasonable for some projects that must rely on certain packages to work.
But if some packages are optional dependencies and can be compiled and used normally even if not, they can be set as optional packages:

add_requires("tbox", {optional = true})

Disable system library

With the default setting, xmake will first check whether the system library exists (if the version requirement is not set). If the user does not want to use the system library and the library provided by the third-party package management at all, then you can set:

add_requires("tbox", {system = false})

Use the debug version of the package

If we want to debug the dependent packages at the same time, we can set to use the debug version of the package (of course, the premise is that this package supports debug compilation):

add_requires("tbox", {debug = true})

If the current package does not support debug compilation, you can submit a modification to the compilation rules in the warehouse to support debugging, for example:

package("openssl")
    on_install("linux", "macosx", function (package)
        os.vrun("./config %s --prefix=\"%s\"", package:debug() and "--debug" or "", package:installdir())
        os.vrun("make -j4")
        os.vrun("make install")
    end)

Pass additional compilation information to the package

Some packages have various compilation options during compilation, and we can also pass them in. Of course, the package itself must support:

add_requires("tbox", {config = {small=true}})

Pass it --small=trueto the tbox package so that the compiled and installed tbox package is enabled with this option.

Use self-built private package warehouse

If the required package is not in the official repository xmake-repo , we can submit the contributed code to the repository for support.
But if some packages are only used for personal or private projects, we can create a private warehouse repo, the organization structure of the warehouse can refer to: xmake-repo

For example, now we have a private warehouse repo:[email protected]:myrepo/xmake-repo.git

We can add the warehouse through the following command:

$ xmake repo --add myrepo [email protected]:myrepo/xmake-repo.git

Or we can write it directly in xmake.lua:

add_repositories("my-repo [email protected]:myrepo/xmake-repo.git")

If we just want to add one or two private packages, it is too trivial to create a git repo specifically at this time, we can put the package warehouse directly in the project, for example:

projectdir
  - myrepo
    - packages
      - t/tbox/xmake.lua
      - z/zlib/xmake.lua
  - src
    - main.c
  - xmake.lua

The myrepo directory above is your own private package warehouse, built into your project, and then add the warehouse location in xmake.lua:

add_repositories("my-repo myrepo")

This can refer to the benchbox project, which has a built-in private warehouse.

We can even define the package description directly in the project xmake.lua without even building a warehouse. This is useful for situations that depend on one or two packages, for example:

package("libjpeg")

    set_urls("http://www.ijg.org/files/jpegsrc.$(version).tar.gz")

    add_versions("v9c", "650250979303a649e21f87b5ccd02672af1ea6954b911342ea491f351ceb7122")

    on_install("windows", function (package)
        os.mv("jconfig.vc", "jconfig.h")
        os.vrun("nmake -f makefile.vc")
        os.cp("*.h", package:installdir("include"))
        os.cp("libjpeg.lib", package:installdir("lib"))
    end)

    on_install("macosx", "linux", function (package)
        import("package.tools.autoconf").install(package)
    end)

package_end()

add_requires("libjpeg")

target("test")
    set_kind("binary")
    add_files("src/*.c") 
    add_packages("libjpeg")

Package management command usage

Package management commands $ xmake requirecan be used to manually display download, compile, install, uninstall, retrieve, and view package information.

Install the specified package

$ xmake require tbox

Install the specified version package:

$ xmake require tbox "~1.6"

Force to download and install again, and display detailed installation information:

$ xmake require -f -v tbox "1.5.x"

Pass additional setting information:

$ xmake require --extra="debug=true,config={small=true}" tbox

Install the debug package, and pass small=truethe compiled configuration information to the package.

Uninstall the specified package

$ xmake require --uninstall tbox

This will completely uninstall and delete the package file.

Remove specified package

Only the unlink specified package is not detected by the current project, but the package still exists locally, and it will be completed quickly if it is reinstalled.

$ xmake require --unlink tbox

View package details

$ xmake require --info tbox

Search for packages in the current warehouse

$ xmake require --search tbox

This is to support fuzzy search and lua pattern matching search:

$ xmake require --search pcr

Pcre, pcre2 and other packages will be searched at the same time.

List currently installed packages

$ xmake require --list

Warehouse management command usage

As mentioned above briefly, adding a private warehouse can be used (supports local path addition):

$ xmake repo --add myrepo [email protected]:myrepo/xmake-repo.git

We can also remove an installed warehouse:

$ xmake repo --remove myrepo

Or view all the added warehouses:

$ xmake repo --list

If the remote warehouse is updated, you can manually perform the warehouse update to get more and the latest packages:

$ xmake repo -u

Submit the package to the official warehouse

At present, this feature has just been completed, and there are not many packages in the official warehouse. Some packages may not support some platforms, but this is not a big problem. After I iterate several versions later, I will continue to expand and improve the package warehouse.

If the package you need is not included in the current official repo , you can submit issues or you can adjust it locally and submit your contribution to the official repo : xmake-repo

For detailed contribution description, see: CONTRIBUTING.md

For more instructions on remote dependency packages, you can see the official document: Remote dependency mode

In fact, the package management of xmake has gone through three generations. The first two versions of v1.0 and v2.0 are local package management mode and system library search mode. These two are still very useful in some cases.
Regarding the introduction of the two, I won't say much here, you can read the document description: Dependent package management

Conclusion

Having said that, let's finally take a look at some other new features and updates provided by the new version:

Other new features

• Added fasm assembler support
• Add has_config, get_configand is_configinterface to quickly determine option and configuration values
• Add set_configinterface to set default configuration
• Add $xmake --tryto try to build the project
• Add set_enabled(false)to display the disabled target
• #69 : Add remote dependency package management,add_requires("tbox ~1.6.1")
• #216 : Add windows mfc compilation rules

Improve

• Improve Qt compilation environment detection, add support for mingw sdk
• Add default debug/release rules in xmake.lua generated by automatic scan
• #178 : Modify the target name under the mingw platform
• For add_files()supporting case-insensitive path pattern matching on windows
• Improve detect.sdks.find_qtthe detection of Qt root directory
• #184 : Improve lib.detect.find_packagesupport for vcpkg
• #208 : Improve rpath's support for dynamic libraries

Bugs fixed

• #177 : Fix the dependent dynamic library target, if the basename is set, the link fails
• Repair $xmake f --menuthe Exit problem and the problem of too high cpu
• #197 : Fix the problem that the generated vs201x project file has garbled Chinese paths
• Fix the blue screen problem when the driver generated by WDK rules compilation runs under Win7
• #205 : Fix the problem that the targetdir and objectdir path settings do not match when the vcproj project generates targetdir

Original source: http://tboox.org/cn/2018/10/13/xmake-update-v2.2.2-package-deps/