Sonar introduction, Ubuntu 18.0.4 SonarQube installation, use, error summary

1. Introduction to Sonar

Sonar (SonarQube) is an open source platform for managing the quality of source code. Sonar is not just a quality data reporting tool, but also a code quality management platform. Supported languages ​​include: Java, PHP, C#, C, Cobol, PL/SQL, Flex, etc.

After the feature idea integrates the sonar plug-in, it can find irregularities and loopholes in the code before the code is submitted, even if it is repaired, instead of waiting for the code quality scan after submitting to the warehouse, and then repeating the submission-detection-repair process .

Sonar can currently eliminate the following types of problem points

• 
  Sonar can standardize code writing through code rule detection tools such as PMD, CheckStyle, and Findbugs if it does not follow the code specification
• Potential defects
  Sonar can detect potential defects through code rule detection tools such as PMD, CheckStyle, Findbugs, etc.
• Overly complex code
  If the code is too complex, it will be difficult to change, which will reduce the readability of the code and make it more difficult for developers to understand
• Duplicate code
  Obviously the program contains a lot of copy-pasted code is of low quality, sonar can show the serious duplication in the source code
• Insufficient or redundant comments.
  Without comments, the readability of the code will be poor, and the readability of the program will be greatly reduced. Too many comments will make developers spend too much energy on reading comments, wasting time
• Lack of unit test
  sonar can easily count and display unit test coverage
• Bad design
  can find loops through sonar, display the interdependence between packages and packages, classes and classes, and detect custom architecture rules. Sonar can manage third-party jar packages, and can use LCOM4 to detect the application of single task rules. case, detection coupling

It is from Architecture Design (architecture design), Coding Rule (coding rules), Potential Bugs (potential errors), Duplications (repetitive code), Comments (comment), Unit Tests (unit test), Complexity (complexity) 7 dimensions Check code quality. Compared with the lint tool, the detection dimension is more comprehensive, and it has a visual and friendly interface for displaying code defects. Combined with CI/CD tools, it can check the code regularly without relying on manual inspection.

Two, Sonar composition

The sonarqube system is a code quality inspection tool consisting of the following four components

A sonarqube server, including three sub-processes (web service (interface management), search service, computing engine service (write to the database)).
A sonarqube database, configure the sonarqube service.
Multiple sonarqube plugins are located in the extensions\plugins directory of the decompression directory.
One or more sonarqube scanners are used to analyze specific projects, which are equivalent to clients.

Workflow
The following architecture diagram is the workflow diagram of the sonar project:

• Developers code in their IDE and use SonarLint to run local analysis.
• Developers push their code into their favorite SCM: git, SVN, TFVC, etc.
• The Continuous Integration Server triggers automatic builds and executes the SonarScanner needed to run SonarQube analysis.
• Analysis reports will be sent to SonarQube server for processing.
• SonarQube Server processes the analysis report results and stores them in the SonarQube database and displays the results in the UI.
• Developers review, comment, challenge their issues through the SonarQube UI to manage and reduce technical debt.
• Managers receive reports from analytics. Ops uses the API to automate configuration and extract data from SonarQube. Operations and maintenance personnel use JMX to monitor SonarQube Server.

Three, SonarQube, JDK, MySQL version corresponding description

Document address: https://docs.sonarqube.org/latest/requirements/prerequisites-and-overview/

SonarQube has version correspondence with JDK and MySQL, check SonarQube correspondence with JDK and database version:

For example, SonarQube 9.8 corresponds to JDK 17 and PostgreSQL 15.

SonarQube 7.6, corresponding to JDK8, MySQL version >=5.6 && <8.0. (Sonarqube no longer supports mysql from 7.8 onwards) (To install the correct version, for example, mysql requires 5.6 or 5.7. If you use 8.0, it will fail. Personal test –)

When you install, you must pay attention, I have stepped on the pit here

If the local jdk is below 8, install 7.8 and below

Four, Ubuntu 18.0.4 SonarQube-7.1.x installation tutorial

zip install

Version Note: The highest sonarqube version supported by jdk8 is 7.8, and jdk11 is required from version 7.9 onwards. When downloading, you need to pay attention to the corresponding relationship between versions, and don't take detours

1. Zip installation download address

• Download address of the latest version of sonarQube: (the latest version does not support mysql) https://www.sonarsource.com/products/sonarqube/downloads/
• Version 7.1 download address: https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-7.1.zip
• Sonar-Scanner 2.5 download address:​ ​https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/​​

Unable to download: refer to the download in this address:
https://blog.51cto.com/u_14082075/5465953#1__6

2. Environment preparation

(1) Java environment (1.8+)

(2) MySql database (5.6+)

（3）SonarQube 7.1

Note: Mysql is no longer supported after sonarQube 7.9, so version 7.1 is downloaded.

3. Upload to ubuntu

//解压
unzip sonarqube-6.7.7.zip
//移动目录
mv sonarqube-6.7.7 /usr/local/sonar

4. Database configuration

.Enter the mysql container and create a user account password is sonar

# docker exec -it mysql bash
# mysql -u root -p
# create database sonar；

# CREATE USER 'sonar'@'%' IDENTIFIED WITH mysql_native_password BY 'sonar';

# GRANT ALL PRIVILEGES ON *.* TO 'sonar'@'%';
# exit

5. Configure the database information of sonar (vim conf/sonar.properties)

The configuration file in conf in the installation directory,

sonar.jdbc.url=jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance&useSSL=false
sonar.jdbc.username=sonar
sonar.jdbc.password=sonar

6. start

Cannot start with root user, create a new user

//新建用户
useradd sonar
//赋权
chown -R sonar.sonar /usr/local/sonar
//修改密码
passwd sonar
//切换到sonar用户
su sonar
//启动： 进入到sonar目录下，即bin的同级目录，执行下边的命令
./bin/linux-x86-64/sonar.sh console

7. Log in to the system

http://ip:9000 username and password: admin
Chinese, just restart the service

5. After sonarqube is installed, Sinicization

If the error "Error while downloading plugin 'l10nzhtw' with version '1.0'. No compatible plugin found." appears, it means that the version is not compatible. You can find the corresponding version of the plugin on the official website and put it in the .../.../extensions/plugins directory. Restart the sonar service [using the command .../.../sonar.sh start or through the page operation "Configuration -> System -> Restart Server"], and it will take effect. But if the installed plug-in is lower than the current version, part of the display will still be in English.

Sinicization plug-in download [find the corresponding version and download the corresponding plug-in]: https://github.com/xuhuisheng/sonar-l10n-zh/releases?page=2

Put it under this path:

cd /usr/local/sonarqube-8.6.0/extensions/plugins

Just restart the service [do not use the root user to log in]

• Generate a token (you need to copy the generated token):
  

After the startup is successful, the corresponding database will be created in the corresponding sonar

6. Sonar scans the gradle project

1. Download the gradle project on git

This is my project directory

2. Configuration

buildscript {
    
    
    repositories {
    
    
        maven {
    
    
            url "http://xxxxxxxxxx"
        }
    }
    dependencies {
    
    
        classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
        // 添加依赖信息
        classpath("org.sonarsource.scanner.gradle:sonarqube-gradle-plugin:2.6.2")
    }

}

// 声明插件
apply plugin: "org.sonarqube"

//配置sonar的配置信息
sonarqube {
    
    
    properties {
    
    
        property "sonar.host.url", "http://127.0.0.1:9000"
        property "sonar.login", "admin"
        property "sonar.password", "admin"
        property "sonar.scm.provider", "git"
        property "sonar.projectKey", artifactId
        property "sonar.projectVersion", version
        property "sonar.sourceEncoding", defaultCharset
        property "sonar.sources", "src/main/java"
        //property "sonar.tests", "src/test/java"
        property "sonar.java.source", "1.8"
        property "sonar.java.binaries", "build/classes/java/main/"
        //property "sonar.java.test.binaries", "build/classes/java/test/"

        property "sonar.java.coveragePlugin", "jacoco"
        property "sonar.coverage.jacoco.xmlReportPaths", "build/reports/jacoco/test/jacocoTestReport.xml"

        property "sonar.dependencyCheck.xmlReportPath", "build/reports/dependency-check-report.xml"
        property "sonar.dependencyCheck.jsonReportPath", "build/reports/dependency-check-report.json"
        property "sonar.dependencyCheck.htmlReportPath", "build/reports/dependency-check-report.html"
    }
}

3. Execute gradle sonarqube

gradle sonarqube

4. Check whether there is scanning information on the web page

7. Export the sonar detection results to PDF

1. Download the PDF export plugin

https://gitee.com/zzulj/sonar-pdf-plugin

Find the corresponding sonarqube version to download, just download the jar directly here, if you download the source code, you need to compile it

2. Copy sonar-pdfreport-plugin-xxxxx.jar to the plugins of sonarqube's extensions.

3. Restart sonarqube, you can see a PDF Report on the web, you need to configure the PDF plug-in, first on the web page, as shown in the figure below, after configuring the password, there is still a userName to configure under the web page, all I configure here are admin

4. After the web page is configured, you need to configure the pdf-plugin in sonarqube

vi  /usr/local/sonar/conf/sonar.properties 

# 在该文件中增加
-Dsonar.pdf.password=admin
-Dsonar.pdf.username=admin

5. After completing some configurations, scan the project again. During the build process, the relevant information of pdf-plugin will appear, and then you can export the PDF on the web page according to the steps in the figure below.

Eight, ide integrated sonar

SonarLint: Sonar plugin for IDEA

1. Search for the "SonarLint" plug-in in the plug-in warehouse, find the plug-in and click Install

2. After the plug-in installation is complete, right-click the project to display SonarLint and select "Analyze with SonarLint"

Nine, error summary

Mistake 1: Elasticsearch virtual memory requires at least [262144]

# docker logs sonarqube_sonarqube_1
ERROR: [1] bootstrap checks failed. You must address the points described in the following [1] lines before starting Elasticsearch.
bootstrap check failure [1] of [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
ERROR: Elasticsearch did not exit normally - check the logs at /opt/sonarqube/logs/sonarqube.log

solution:

After a rough look, it is because the minimum memory consumption of ElasticSearch needs to reach 262144 when running, so if it is running on your own virtual machine, you can temporarily adjust the size of vm.max_map_count. The max_map_count file contains the VMA (virtual The number of memory areas), the temporary adjustment statement is as follows:
sysctl -w vm.max_map_count=262144
After adjustment, there will be no problem running the es cluster.
If you want to permanently modify the size of vm.max_map_count, you can use the following command:
echo vm.max_map_count=262144>> /etc/sysctl.conf sysctl -p

Mistake 2: Starting with the root account

Cannot start with root user, create a new user, I created a new sonar user, then authorize the user, start

10. Unresolved problems [docker installation]

In the beginning, to save trouble, I used docker to install, but it has been unable to start, and the error Can not connect to database. Please check connectivity and settings (see the properties prefixed by 'sonar.jdbc.').

According to the solutions provided on the Internet, none of them worked

• 1. The database user name and password use sonar/sonar database name: sonar

Docker installation steps [unsuccessful]

1...docker pulls the sonarqube image

docker pull sonarqube:7.8-community

You can find the version you want on the official website: https://hub.docker.com/_/sonarqube?tab=tags&page=9

2... Enter the mysql container and create a user account password is sonar

# docker exec -it mysql bash
# mysql -u root -p
# create database sonar；

# CREATE USER 'sonar'@'%' IDENTIFIED WITH mysql_native_password BY 'sonar';

# GRANT ALL PRIVILEGES ON *.* TO 'sonar'@'%';
# exit

3. Start sonarqube

docker run -d --name sonar -p 9000:9000 -p 9092:9092 -v /data/sonar/conf:/opt/sonarqube/conf -v /data/sonar/data:/opt/sonarqube/data -v /data/sonar/logs:/opt/sonarqube/logs -v /data/sonar/extensions:/opt/sonarqube/extensions -e "SONARQUBE_DATABASE_USER=sonar" -e "SONARQUBE_DATABASE_NAME=sonar" -e "SONARQUBE_DATABASE_PASSWORD=sonar"  -e "SONARQUBE_JDBC_URL=jdbc:mysql://127.0.0.1:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance&useSSL=false" sonarqube:7.8-community

4. Whether the database configuration in the corresponding mapping configuration/data/sonar/conf is correct

vi sonar.properties

#sonar.jdbc.url=jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformanc
e#sonar.jdbc.username=sonar
#sonar.jdbc.password=sonar
#sonar.sorceEncoding=UTF-8

root@koal-ipsec:/data/sonar/conf# docker logs sonar 
2022.12.26 05:15:07 INFO  app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /opt/sonarqube/temp
2022.12.26 05:15:07 INFO  app[][o.s.a.es.EsSettings] Elasticsearch listening on /127.0.0.1:9001
2022.12.26 05:15:07 INFO  app[][o.s.a.p.ProcessLauncherImpl] Launch process[[key='es', ipcIndex=1, logFilenamePrefix=es]] from [/opt/sonarqube/elasticsearch]: /opt/sonarqube/elasticsearch/bin/elasticsearch -Epath.conf=/opt/sonarqube/temp/conf/es
2022.12.26 05:15:07 INFO  app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
2022.12.26 05:15:07 INFO  app[][o.e.p.PluginsService] no modules loaded
2022.12.26 05:15:07 INFO  app[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.transport.Netty4Plugin]
2022.12.26 05:15:14 INFO  app[][o.s.a.SchedulerImpl] Process[es] is up
2022.12.26 05:15:14 INFO  app[][o.s.a.p.ProcessLauncherImpl] Launch process[[key='web', ipcIndex=2, logFilenamePrefix=web]] from [/opt/sonarqube]: /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/opt/sonarqube/temp -Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError -Djava.security.egd=file:/dev/./urandom -cp ./lib/common/*:./lib/server/*:/opt/sonarqube/lib/jdbc/mysql/mysql-connector-java-5.1.42.jar org.sonar.server.app.WebServer /opt/sonarqube/temp/sq-process8004789798386622755properties
2022.12.26 05:15:14 INFO  web[][o.s.p.ProcessEntryPoint] Starting web
2022.12.26 05:15:15 INFO  web[][o.a.t.u.n.NioSelectorPool] Using a shared selector for servlet write/read
2022.12.26 05:15:15 INFO  web[][o.e.p.PluginsService] no modules loaded
2022.12.26 05:15:15 INFO  web[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.index.reindex.ReindexPlugin]
2022.12.26 05:15:15 INFO  web[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.join.ParentJoinPlugin]
2022.12.26 05:15:15 INFO  web[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.percolator.PercolatorPlugin]
2022.12.26 05:15:15 INFO  web[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.transport.Netty4Plugin]
2022.12.26 05:15:16 INFO  web[][o.s.s.e.EsClientProvider] Connected to local Elasticsearch: [127.0.0.1:9001]
2022.12.26 05:15:16 INFO  web[][o.s.s.p.LogServerVersion] SonarQube Server / 6.7.5.38563 / ec8ab77fa180a9edaf664075e8e76ae0b4a2d9f1
2022.12.26 05:15:16 INFO  web[][o.sonar.db.Database] Create JDBC data source for jdbc:mysql://127.0.0.1:3306/db_sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance&useSSL=false
2022.12.26 05:15:16 ERROR web[][o.s.s.p.Platform] Web server startup failed
java.lang.IllegalStateException: Can not connect to database. Please check connectivity and settings (see the properties prefixed by 'sonar.jdbc.').
        at org.sonar.db.DefaultDatabase.checkConnection(DefaultDatabase.java:108)
        at org.sonar.db.DefaultDatabase.start(DefaultDatabase.java:75)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.picocontainer.lifecycle.ReflectionLifecycleStrategy.invokeMethod(ReflectionLifecycleStrategy.java:110)
        at org.picocontainer.lifecycle.ReflectionLifecycleStrategy.start(ReflectionLifecycleStrategy.java:89)
        at org.picocontainer.injectors.AbstractInjectionFactory$LifecycleAdapter.start(AbstractInjectionFactory.java:84)
        at org.picocontainer.behaviors.AbstractBehavior.start(AbstractBehavior.java:169)
        at org.picocontainer.behaviors.Stored$RealComponentLifecycle.start(Stored.java:132)
        at org.picocontainer.behaviors.Stored.start(Stored.java:110)
        at org.picocontainer.DefaultPicoContainer.potentiallyStartAdapter(DefaultPicoContainer.java:1016)
        at org.picocontainer.DefaultPicoContainer.startAdapters(DefaultPicoContainer.java:1009)
        at org.picocontainer.DefaultPicoContainer.start(DefaultPicoContainer.java:767)
        at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:134)
        at org.sonar.server.platform.platformlevel.PlatformLevel.start(PlatformLevel.java:90)
        at org.sonar.server.platform.Platform.start(Platform.java:211)
        at org.sonar.server.platform.Platform.startLevel1Container(Platform.java:170)
        at org.sonar.server.platform.Platform.init(Platform.java:86)
        at org.sonar.server.platform.web.PlatformServletContextListener.contextInitialized(PlatformServletContextListener.java:45)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4745)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5207)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1419)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1409)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.)
        at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1549)
        at org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1388)
        at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
        at org.sonar.db.profiling.NullConnectionInterceptor.getConnection(NullConnectionInterceptor.java:31)
        at org.sonar.db.profiling.ProfiledDataSource.getConnection(ProfiledDataSource.java:323)
        at org.sonar.db.DefaultDatabase.checkConnection(DefaultDatabase.java:106)
        ... 29 common frames omitted
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:989)
        at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:341)
        at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2192)
        at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2225)
        at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2024)
        at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:779)
        at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:389)
        at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330)
        at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
        at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
        at org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(BasicDataSource.java:1556)
        at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1545)
        ... 34 common frames omitted
Caused by: java.net.ConnectException: Connection refused (Connection refused)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:211)
        at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:300)
        ... 50 common frames omitted
2022.12.26 05:15:17 INFO  app[][o.s.a.SchedulerImpl] Process [web] is stopped
2022.12.26 05:15:17 WARN  app[][o.s.a.p.AbstractProcessMonitor] Process exited with exit value [es]: 143
2022.12.26 05:15:17 INFO  app[][o.s.a.SchedulerImpl] Process [es] is stopped
2022.12.26 05:15:17 INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped