【Doris】Doris数据库最新版安装方法，详细图文教程

环境安装

Doris 作为一款开源的 MPP 架构 OLAP 数据库，能够运行在绝大多数主流的商用服务器上。为了能够充分运用 MPP 架构的并发优势，以及 Doris 的高可用特性，我们建议 Doris 的部署遵循以下需求：

Linux 操作系统	版本需求
Linux 系统	版本
CentOS	7.1 及以上
Ubuntu	16.04 及以上

软件需求

软件	版本
Java	1.8 及以上
GCC	4.8.2 及以上

操作系统安装要求

设置系统最大打开文件句柄数

vi /etc/security/limits.conf 
* soft nofile 65536
* hard nofile 65536

时钟同步

Doris 的元数据要求时间精度要小于5000ms，所以所有集群所有机器要进行时钟同步，避免因为时钟问题引发的元数据不一致导致服务出现异常。

关闭交换分区（swap）

Linux交换分区会给Doris带来很严重的性能问题，需要在安装之前禁用交换分区。
查看交换分区是否开启：

free -m

若开启则修改配置项，注释此行：

#/swapfile                                 none            swap    sw              0       0

然后reboot重启。

Liunx文件系统

这里我们推荐使用ext4文件系统，在安装操作系统的时候，请选择ext4文件系统。

开发测试环境

模块	CPU	内存	磁盘	网络	实例数量
Frontend	8核+	8GB+	SSD 或 SATA，10GB+ *	千兆网卡	1
Backend	8核+	16GB+	SSD 或 SATA，50GB+ *	千兆网卡	1-3 *

生产环境

模块	CPU	内存	磁盘	网络	实例数量（最低要求）
Frontend	16核+	64GB+	SSD 或 RAID 卡，100GB+ *	万兆网卡	1-3 *
Backend	16核+	64GB+	SSD 或 SATA，100G+ *	万兆网卡	3 *

java环境

命令行输入jps，若报错-bash: jps: command not found，则说明jdk需要重装。若已正常安装jdk，则跳过本节内容。
输入命令

whereis java

这只是java的执行路径，而非java的安装路径。
输入命令

which java

这个路径是java路径的软链接，获取该链接指向的地址：

ls -l /usr/bin/java

继续查询真实地址：

ls -l /etc/alternatives/java

发现路径是CentOS自带的openjdk，所以我们需要在oracle官网下载jdk1.8重新安装，首先卸载原有的openjdk：

rpm -qa | grep java #查看java文件
sudo rpm -qa | grep java | xargs rpm -e --nodeps #root权限下删除所有java文件

jdk下载官网：https://www.oracle.com/java/technologies/downloads/#java8-linux

下载最下面的版本，解压：

tar -zxvf jdk-8u351-linux-x64.tar.gz

linux目录下：
/usr/local 文件一般存放给另一个主机存放额外安装软件的目录，一般是通过编译源码方式安装的程序，如jdk、nginx、rabbitmq；
/opt 则是存放额外安装软件的目录，如mysql、redis、zookeeper；
/usr 用户的很多应用程序和文件都放在这个目录里面

配置java环境变量：

vim /etc/profile

在末尾追加：

export JAVA_HOME=/usr/local/jdk/jdk1.8.0_351  #jdk安装目录
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib:$CLASSPATH
export JAVA_PATH=${JAVA_HOME}/bin:${JRE_HOME}/bin
export PATH=$PATH:${JAVA_PATH}

刷新profile文件：

source /etc/profile

检查是否安装成功：

java -version
javac

输入命令行jps：

此时即为正常。

GCC环境

检查GCC是否安装，并保证版本在4.8.2 及以上：

a@a:~$ gcc

Command 'gcc' not found, but can be installed with:

sudo apt install gcc

a@a:~$ sudo apt install gcc   # 安装GCC

a@a:~$ gcc -v   # 查看GCC版本
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/9/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:hsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 9.4.0-1ubuntu1~20.04.1' --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-9 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-9-Av3uEd/gcc-9-9.4.0/debian/tmp-nvptx/usr,hsa --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)

mysql安装

doris采用的mysql协议，需要通过mysql来连接使用，没有mysql的话需要先安装mysql：

mysql
sudo apt install mysql-client-core-8.0     # version 8.0.30-0ubuntu0.20.04.2, or
sudo apt install mariadb-client-core-10.3  # version 1:10.3.34-0ubuntu0.20.04.1

安装包下载

首先查询cpu型号：

cat /proc/cpuinfo

若cpu支持avx2，则直接下载avx2下的三个文件。
首先解压FE和BE， 然后第三个包java-udf-jar-with-dependencies.jar放到 apache-doris-be-1.2.1-bin-x86_64/lib 目录下。包位置放错，后续BE会启动不了。
设置环境变量：

vim /etc/profile.d/doris.sh

FE

配置FE：

a@a:/opt/doris/apache-doris-fe-1.2.1-bin-x86_64/conf # pwd
a@a:/export/server/apache-doris-1.1.1-bin-x86/fe/conf # ll
# vim fe.conf
root@zhiyong-doris:/export/server/apache-doris-1.1.1-bin-x86/fe/conf# cat fe.conf
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.

#####################################################################
## The uppercase properties are read and exported by bin/start_fe.sh.
## To see all Frontend configurations,
## see fe/src/org/apache/doris/common/Config.java
#####################################################################

# the output dir of stderr and stdout
LOG_DIR = ${DORIS_HOME}/log

DATE = `date +%Y%m%d-%H%M%S`
JAVA_OPTS="-Xmx8192m -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xloggc:$DORIS_HOME/log/fe.gc.log.$DATE"

# For jdk 9+, this JAVA_OPTS will be used as default JVM options
JAVA_OPTS_FOR_JDK_9="-Xmx8192m -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xlog:gc*:$DORIS_HOME/log/fe.gc.log.$DATE:time"

##
## the lowercase properties are read by main program.
##

# INFO, WARN, ERROR, FATAL
sys_log_level = INFO

# store metadata, must be created before start FE.
# Default value is ${DORIS_HOME}/doris-meta
meta_dir = ${DORIS_HOME}/doris-meta

http_port = 8030
rpc_port = 9020
query_port = 9030
edit_log_port = 13536
mysql_service_nio_enabled = true

# Choose one if there are more than one ip except loopback address.
# Note that there should at most one ip match this list.
# If no ip match this rule, will choose one randomly.
# use CIDR format, e.g. 10.10.10.0/24
# Default value is empty.
# priority_networks = 10.10.10.0/24;192.168.0.0/16
priority_networks = 192.168.88.0/24

# Advanced configurations
# log_roll_size_mb = 1024
# sys_log_dir = ${DORIS_HOME}/log
# sys_log_roll_num = 10
# sys_log_verbose_modules = org.apache.doris
# audit_log_dir = ${DORIS_HOME}/log
# audit_log_modules = slow_query, query
# audit_log_roll_num = 10
# meta_delay_toleration_second = 10
# qe_max_connection = 1024
# max_conn_per_user = 100
# qe_query_timeout_second = 300
# qe_slow_log_ms = 5000

其中，edit_log_port的9010端口容易冲突，需注意。此外，${DORIS_HOME}/doris-meta目录需要提前创建好。

启动FE

FE的bin路径下执行：

./start_fe.sh --daemon  
jps # 查看FE是否启动，若有PaloFe，则说明FE启动成功

BE

修改BE配置文件夹，同理，修改BE的conf文件夹下的be.conf:

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.

PPROF_TMPDIR="$DORIS_HOME/log/"

# INFO, WARNING, ERROR, FATAL
sys_log_level = INFO

# ports for admin, web, heartbeat service
be_port = 9060
webserver_port = 13535
heartbeat_service_port = 9050
brpc_port = 8060

# Choose one if there are more than one ip except loopback address.
# Note that there should at most one ip match this list.
# If no ip match this rule, will choose one randomly.
# use CIDR format, e.g. 10.10.10.0/24
# Default value is empty.
# priority_networks = 10.10.10.0/24;192.168.0.0/16
priority_networks = 192.168.88.0/24

# data root path, separate by ';'
# you can specify the storage medium of each root path, HDD or SSD
# you can add capacity limit at the end of each root path, seperate by ','
# eg:
# storage_root_path = /home/disk1/doris.HDD,50;/home/disk2/doris.SSD,1;/home/disk2/doris
# /home/disk1/doris.HDD, capacity limit is 50GB, HDD;
# /home/disk2/doris.SSD, capacity limit is 1GB, SSD;
# /home/disk2/doris, capacity limit is disk capacity, HDD(default)
#
# you also can specify the properties by setting '<property>:<value>', seperate by ','
# property 'medium' has a higher priority than the extension of path
#
# Default value is ${DORIS_HOME}/storage, you should create it by hand.
# storage_root_path = ${DORIS_HOME}/storage
storage_root_path = /opt/doris/storage

# Advanced configurations
# sys_log_dir = ${DORIS_HOME}/log
# sys_log_roll_mode = SIZE-MB-1024
# sys_log_roll_num = 10
# sys_log_verbose_modules = *
# log_buffer_level = -1
# palo_cgroups

提前建好storage文件夹目录。
如果 be 部署在 hadoop 集群中，注意调整 be.conf 中的 webserver_port。

在 FE 中添加所有 BE 节点

连接doris：

mysql -h 192.168.88.21 -P 9030 -uroot
mysql> show databases;
mysql> use information_schema;
mysql> show tables;
mysql> ALTER SYSTEM ADD BACKEND "192.168.88.21:9050";

启动BE

BE的bin目录下输入：

./start_be.sh --daemon

查看BE状态：

mysql> SHOW PROC '/backends';

若Alive项为true则说明BE启动成功，若为false则说明启动失败。
启动失败可以检查doris-meta和storage文件夹是否提前建立，端口是否冲突，一般错误都是这两个原因引起的，错误往往是由于填写的配置信息有问题。