Google multi-source management tool gclient

Remarks: This article is excerpted from https://blog.csdn.net/cabbage2008/article/details/52766821. I am very grateful to the blogger for sharing. This reprint is only for learning records. If there is any infringement, please inform me in time, and I will delete it in time.

This article is organized according to the help of gclient.

Google's chromium project uses gclient to manage source code checkout, update, etc. gclient is a script specially written by google for this kind of multi-source project. It can manage the codes in multiple source management systems together. Even including putting git and svn code together.

Gclient's sync, update and other commands are easy to learn and use. I won't say more, but focus on two types of files closely related to gclient. gclient and DEPS.

The .gclient file is the gclient control file, which is placed at the top of the working directory. The ".gclient" file is a Python script (Google really has a soft spot for python), which defines a set of "solutions" in a format similar to the following

[python] view plain copy

1. solutions = [  
2.   { "name"        : "src",  
3.     "url"         : "svn://svnserver/component/trunk/src",  
4.     "custom_deps" : {  
5.       # To use the trunk of a component instead of what's in DEPS:  
6.       #"component": "https://svnserver/component/trunk/",  
7.       # To exclude a component from your working copy:  
8.       #"data/really_large_component": None,  
9.     }  
10.   },  
11. ]  

• name : the name of the checkout source code
• url : The directory where the source code is located, gclient expects the source code to be checked out to include a DEPS file, which contains the information of the source code that must be checked out to the working directory;
• deps_file This is a file name (excluding the path), which refers to the file containing the dependency list in the project directory. This item is optional and the default value is "DEPS"
• custom_deps This is an optional dictionary object that overrides the entries defined in the project's "DEPS" file. Generally it is used in the local directory, those code without checkout, such as

[python] view plain copy

1. "custom_deps": {  
2.   "src/content/test/data/layout_tests/LayoutTests": None,  
3.   "src/chrome/tools/test/reference_build/chrome_win": None,  
4.   "src/chrome_frame/tools/test/reference_build/chrome_win": None,  
5.   "src/chrome/tools/test/reference_build/chrome_linux": None,  
6.   "src/chrome/tools/test/reference_build/chrome_mac": None,  
7.   "src/third_party/hunspell_dictionaries": None,  
8. },  

         Or let the local directory checkout a new code from a different location, or checkout a different branch, version, etc. Can also be used to add new items that do not exist in DEPS

• target_os : This optional entry can point to a specific platform, and check out different codes according to the platform, such as

[python] view plain copy

1. target_os = ['android']  

If the target_os_only value is True, then only the corresponding code is checked out, such as

[python] view plain copy

1. target_os = [ "ios" ]  
2. target_os_only = True  

In each checkout project, gclient expects to find a DEPS file (given by deps_file), which defines how different parts of the project are checked out.

"DEPS" is also a python script, the simplest, as follows:

[python] view plain copy

1. deps = {  
2.   "src/outside" : "http://outside-server/trunk@1234",  
3.   "src/component" : "svn://svnserver/component/trunk/src@77829",  
4.   "src/relative" : "/trunk/src@77829",  
5. }  

Each entry in deps contains a key-value pair, where the key is the local directory being checked out, and the value is the corresponding remote URL.

If the path starts with '/', then it is a relative URL, relative to the URL in .gclient.

The URL usually includes a version number in order to lock the source code to a specific version. Of course, this is optional. If not, then it will fetch the latest version on the specified branch.

DEPS can also contain other types of data such as vars, 

[python] view plain copy

1. vars = {  
2.   'pymox':  
3.     'http://pymox.googlecode.com/svn',  
4.   'sfntly':  
5.     'http://sfntly.googlecode.com/svn',  
6.   'eyes-free':  
7.     'http://eyes-free.googlecode.com/svn',  
8.   'rlz':  
9.     'http://rlz.googlecode.com/svn',  
10.   'smhasher':  
11.     'http://smhasher.googlecode.com/svn',  
12. ...  
13. }  

Vars defines a set of variables , which can be accessed through Var(xxx) later. Var(xxx) returns a string, so operations can also be performed, such as

[python] view plain copy

1.      'src/third_party/cros_dbus_cplusplus/source':  
2.      Var("git.chromium.org") + '/chromiumos/third_party/dbus-cplusplus.git@5e8f6d9db5c2abfb91d91f751184f25bb5cd0900',  
3.    'src/third_party/WebKit':  
4. nbsp;     Var("webkit_trunk")[:-6] + '/branches/chromium/1548@153044',  

The second one is independent, Var("webkit_trunk")[:-6] is a python expression, which means to get the last 6 characters of the string represented by "webkit_trunk"

Hooks: DEPS contains optional content hooks, which also play an important role. It means that a hook operation is performed after sync, update or recert.

If the --nohooks option is used (hook is executed by default), then the hook will not be executed after gclient sync or other operations. You can do this individually with gclient runhooks; if you have gclient sync --force, the hook will be executed regardless of whether the sync was successful or not.

The way hook is written in DEPS is generally:

[python] view plain copy

1. hooks = [  
2.   { "pattern": "\\.(gif|jpe?g|pr0n|png)$",  
3.     "action":  ["python", "image_indexer.py", "--all"]},  
4.   { "pattern": ".",  
5.     "name": "gyp",  
6.     "action":  ["python", "src/build/gyp_chromium"]},  
7. ]  

hooks contains a set of hooks, each hook has several important items:

• pattern is a regular expression used to match files in the project directory. Once the match is successful, the action item will be executed
• action describes a command line to run based on specific arguments. This command is run at most once per gclient, no matter how many files match. This command is run in the same directory as .gclient. If the first argument is "python", then the current python interpreter will be used. If the string "$matching_files" is included, it expands the string to a list of matched files.
• name is optional and marks the group the hook belongs to, which can be used to override and reorganize.

deps_os: Projects that define different platform dependencies in DEPS, such as

[python] view plain copy

1. deps_os = {  
2.   "win": {  
3.     "src/chrome/tools/test/reference_build/chrome_win":  
4.       "/trunk/deps/reference_builds/chrome_win@197743",  
5.   
6.     "src/third_party/cygwin":  
7.       "/trunk/deps/third_party/cygwin@133786",  
8.   
9. .....  
10.   },  
11.   
12.   "ios": {  
13.     "src/third_party/GTM":  
14.       (Var("googlecode_url") % "google-toolbox-for-mac") + "/trunk@" +  
15.       Var("gtm_revision"),  
16.   
17.     "src/third_party/nss":  
18.       "/trunk/deps/third_party/nss@" + Var("nss_revision"),  
19. ....  
20.    },  
21. ...  
22. }  

deps_os specifies the dependencies of different platforms, which can contain multiple platforms, corresponding to target_os in .gclient. This correspondence is as follows:

[python] view plain copy

1. DEPS_OS_CHOICES = {  
2.   "win32": "win",  
3.   "win": "win",  
4.   "cygwin": "win",  
5.   "darwin": "mac",  
6.   "mac": "mac",  
7.   "unix": "unix",  
8.   "linux": "unix",  
9.   "linux2": "unix",  
10.   "linux3": "unix",  
11.   "android": "android",  
12. }