CodeQL code security scanning tool installation and deployment

CodeQL is an analytics engine used by developers to automate security checks and used by security researchers to perform variant analysis.
In CodeQL, code is treated as data. Security holes, bugs, and other bugs are modeled as queries that can be executed against a database extracted from the code.
You can run standard CodeQL queries written by GitHub researchers and community contributors, or write your own for custom analysis. Queries that find potential errors highlight results directly in the source file.

CodeQL analysis consists of three steps:

1. Prepare your code by creating a CodeQL database
2. Run CodeQL queries against the database
3. interpret query results

Languages ​​supported by CodeQL:

language	Variants	translater	expand
C/C++		Clang (and clang-cl) extensions (up to Clang 12.0) GNU extensions (up to GCC 11.1) Microsoft extensions (up to VS 2019) Arm compiler 5	.cpp, .c++, .cxx, .hpp, .hh, .h++, .hxx, .c, .cc,.h
C＃	C# up to 9.0	Microsoft Visual Studio up to 2019, .NET up to 4.8, .NET Core up to 3.1, .NET 5	.sln, .csproj, .cs, .cshtml,.xaml
Golang	up to 1.16	Go to 1.11 or newer	.go
Java	Java 7 to 16	javac (OpenJDK and Oracle JDK), the Eclipse Compiler for Java (ECJ)	.java
JavaScript	ECMAScript 2021 or earlier	not applicable	.js, .jsx, .mjs, .es, .es6, .htm, .html, .xhtm, .xhtml, .vue, .hbs, .ejs, .njk, .json, .yaml, .yml, .raml, .xml [6]
Python	2.7、3.5、3.6、3.7、3.8、3.9	not applicable	.py
Ruby	up to 3.0.2	not applicable	.rb, .erb, .gemspec,Gemfile
TypeScript	2.6-4.5	Standard TypeScript compiler	.ts, .tsx

Recommended application method:

• Use 1.2 official source code to install.

• Combined with jenkins to automatically detect security in CI.

• View detections and make corrections with VSCode and VisualStudio extensions.

1. Install CodeQL

To install using official source code, you need to package and compile the environment yourself, and enter Dotnet, NodeJS, Npm, etc.

Create the root directory CodeQLHome locally

mkdir CodeQL
cd CodeQL
mkdir codeql-home
cd codeql-home

1.1 Install CLI

Download address: https://github.com/github/codeql-cli-binaries/releases

Version: 2.7.1

wget https://github.com/github/codeql-cli-binaries/releases/download/v2.7.1/codeql-linux64.zip

1.2 Install query library

The query library is a collection of .ql or .qls files for filtering code

Version: 1.29.0

Download address: https://github.com/github/codeql/tags

Among them, "lgtm-xxxx" is the warehouse, because the source code management is very bad, if you can't find the query library, you can directly replace it with the source code.

wget https://github.com/github/codeql/archive/refs/tags/lgtm/v1.29.0.zip

1.3 Integration

• Unzip the CLI package to the CodeQLHome directory and name it codeql

  unzip -o codeql-linux64.zip
  

• Unzip the query library package to the CodeQLHome directory and name it codeql-repo

  unzip -o v1.29.0.zip
  

1.4 configuration

Change the environment variable to point to the CLI home directory, CodeQLHome/codeql

sudo vim /etc/profile

Add the following to the end of the document

#Path CodeQL
export PATH=$PATH:/home/username/CodeQL/codeql-home/codeql

Save the file and exit the editor.

Restart application configuration

source  /etc/profile

Switch to the command line, run

codeql version

Get the correct output and the configuration is complete

See linux configuration: https://www.jianshu.com/p/4274e679dec6

2. Code inspection

2.1 Javascript check

Environment preparation:

Install nodejs, install nestjs (not required), install npm

1) Create a repository

codeql  database create --language=javascript ./projects-result/test3_db -s ./projects-src/testproject

2) Update configuration

codeql database upgrade ./projects-result/test3_db

3) Execute the query

codeql database analyze ./projects-result/test3_db --format=sarifv2.1.0 --output=./projects-result/test3_db/issues.sarif javascript-security-and-quality.qls

2.2 Dotnet check

Environment preparation:

Install dotnetcore 3.1 and dotnet6 runtime.

Add global.jsonto the root directory of the source code, this file can solve the support for .NET 5 6

{
    
    
  "sdk": {
    
    
    "version": "5.0.0",
    "rollForward": "latestMajor"
  }
}

Elevate folder permissions because files are generated when compiling.

chmod -R 777 /home/username/CodeQL/projects-src
cd /home/username/CodeQL

1) Create a repository

codeql  database create --language=csharp ./projects-result/test4_db -s ./projects-src/testproject2

2) Update configuration

codeql database upgrade ./projects-result/test4_db

3) Execute the query

codeql database analyze ./projects-result/test4_db --format=sarifv2.1.0 --output=./projects-result/test4_db/issues.sarif csharp-security-and-quality.qls

3. View the results

Address: https://sarifweb.azurewebsites.net/

4. CI Integration

Address: https://docs.github.com/en/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system

4.1. Jenkins integration

1. Plugin: https://plugins.jenkins.io/codeql/

   It is recommended that jenkins and CodeQL be deployed on the same server, so that the source code only needs to be downloaded once, and at the same time, the script package deployed by the CodeQL container can be conveniently called to simplify operations.

4.2. VSCode integration

• Search for and install the extension codeql

• Extended settings

Executable Path->D:/Software/CodeQLHome/codeql/codeql.exe

• Source code target library generation

​ Assuming that the local code is in the TypeScript program of D:\Projects\Local\Test\npmRes\abc

​ Generate database:

codeql database create --language=javascript D:\Projects\Local\CodeQL\RESULT0127\source_db -s D:\Projects\Local\Test\npmRes\abc

• VSCode specifies the source code library

Enter the small icon at the bottom left of VSCode's CodeQL, select "Select from folder", and set it as the currently available library.

• select workspace

Select D:/Software/CodeQLHome/codeql-repo folder

进入"javascript/ql/src/"->任意*.ql文件右键->CodeQL:Run queries

• View the results on the right