《GreenPlum系列-部署维护》超详细Docker安装GreenPlum单机节点教程

超详细Docker安装GreenPlum单机节点教程

一、创建centos容器

1.1 拉取centos7镜像

docker pull centos:7

[root@zxy ~]# docker pull centos:7
[root@zxy ~]# docker images
REPOSITORY      TAG       IMAGE ID       CREATED         SIZE
centos         7         eeb6ee3f44bd   18 months ago   204MB

1.2 创建容器

外部端口6002映射容器内部ssh端口22

外部端口5432映射内部端口5432

外部端口6000映射内部端口6000

外部端口6001映射内部端口6001

[root@zxy ~]# docker run -itd --name gp-docker01 \
> -p 6002:22 \
> -p 5432:5432 \
> -p 6000:6000 \
> -p 6001:6001 \
> --privileged eeb6ee3f44bd \
> /usr/sbin/init
b0c0fe56c04889c3d22aed9f422e16647030517b97f787095b70a8bf84f757a4

1.3 进入容器

[root@zxy ~]# docker exec -it b0c0fe56c048 /bin/bash
[root@b0c0fe56c048 /]#

1.4 容器和服务器免密操作

1.4.1 生成密钥

如果在执行ssh-keygen的时候，找不到该命令，那么使用yum安装openssh即可

bash: ssh-keygen: command not found

# 1.ssh-keygen失败
[root@b0c0fe56c048 /]# ssh-keygen
bash: ssh-keygen: command not found
# 2.安装openssh
[root@b0c0fe56c048 /]# yum install openssh
# 3.再次进行生成密钥操作，不用输入，直接点击enter键即可
[root@b0c0fe56c048 /]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
......

1.4.2 拷贝密钥

如果在执行ssh-copy-id失败的话，使用yum手动安装openssh-clients即可

bash: ssh-copy-id: command not found

# 1.拷贝失败
[root@b0c0fe56c048 /]# ssh-copy-id 125.22.95.188
bash: ssh-copy-id: command not found
# 2.安装openssh-clients
[root@b0c0fe56c048 /]# yum install openssh-clients
# 3.再次拷贝，进行免密
[root@b0c0fe56c048 /]# ssh-copy-id 125.22.95.188
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host '125.22.95.188 (125.22.95.188)' can't be established.
ECDSA key fingerprint is SHA256:uvxqg9O/HlNw5Y3FNToE/llF8GeZQy/p+GXlIS3N6cY.
ECDSA key fingerprint is MD5:d4:15:a9:ea:f9:26:8f:2f:b4:dd:8a:42:3b:58:29:3b.
Are you sure you want to continue connecting (yes/no)? yes
.....

1.5 安装ssh服务和网络必须应用

[root@b0c0fe56c048 /]# yum -y install net-tools.x86_64
[root@b0c0fe56c048 /]# yum -y install openssh-server
[root@b0c0fe56c048 /]# systemctl restart sshd

1.6 容器设置root密码

1.6.1 安装passwd应用

安装passwd应用，可以给容器的用户设置密码，方便对本机进行免密操作

[root@b0c0fe56c048 /]# yum -y install passwd
Loaded plugins: fastestmirror, ovl
Loading mirror speeds from cached hostfile
 * base: ftp.sjtu.edu.cn
 * extras: ftp.sjtu.edu.cn
 * updates: ftp.sjtu.edu.cn
Package passwd-0.79-6.el7.x86_64 already installed and latest version
Nothing to do

1.6.2 容器本机root设置密码

[root@b0c0fe56c048 /]# passwd root
Changing password for user root.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.

1.7 容器本机免密

[root@b0c0fe56c048 /]# ssh-copy-id localhost
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host 'localhost (127.0.0.1)' can't be established.
ECDSA key fingerprint is SHA256:9iOpCm67LKlvIZyEUNwxF4BnGlvORfRAbZX79nYkW1M.
ECDSA key fingerprint is MD5:fa:38:d7:e3:40:ad:12:68:94:5b:35:8c:77:8b:f0:bb.
Are you sure you want to continue connecting (yes/no)? ys
Please type 'yes' or 'no': yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@localhost's password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'localhost'"
and check to make sure that only the key(s) you wanted were added.

二、安装GreenPlum

2.1 准备安装包

[root@zxy ~]# docker cp /zxy/software/greenplum-db-6.4.0-rhel6-x86_64.rpm b0c0fe56c048:/

2.2 创建用户及用户组

[root@b0c0fe56c048 /]# groupadd gpadmin
[root@b0c0fe56c048 /]# useradd gpadmin -g gpadmin
[root@b0c0fe56c048 /]# passwd gpadmin

2.3 离线安装

2.3.1 检查安装包并yum安装

安装包

官网

官方推荐使用yum的方式安装，yum安装的饿好处是，会自动帮我们下载安装依赖包。默认将greenplum软件安装到/usr/local目录下，并创建软连接。但是如果不能自动联网下载就会比较麻烦。

[root@b0c0fe56c048 /]# ls | grep greenplum
greenplum-db-6.4.0-rhel6-x86_64.rpm
[root@b0c0fe56c048 /]# yum install localhost greenplum-db-6.4.0-rhel6-x86_64.rpm

2.3.2 查看安装路径

[root@b0c0fe56c048 /]# ll /usr/local/ | grep greenplum
lrwxrwxrwx  1 root root   29 Mar 22 02:29 greenplum-db -> /usr/local/greenplum-db-6.4.0
drwxr-xr-x 12 root root 4096 Mar 22 02:29 greenplum-db-6.4.0

2.3.3 将greenplum目录权限授予gpadmin用户

[root@b0c0fe56c048 /]# chown -Rf gpadmin:gpadmin /usr/local/greenplum*
[root@b0c0fe56c048 /]# ll /usr/local/ | grep greenplum
lrwxrwxrwx  1 gpadmin gpadmin   29 Mar 22 02:29 greenplum-db -> /usr/local/greenplum-db-6.4.0
drwxr-xr-x 12 gpadmin gpadmin 4096 Mar 22 02:29 greenplum-db-6.4.0

2.3.4 创建数据目录和配置目录

创建数据目录gpdata有master和primary等节点

创建配置目录gpconfigs

[root@b0c0fe56c048 /]# mkdir -p /data/gpdata/master
[root@b0c0fe56c048 /]# mkdir -p /data/gpdata/primary
[root@b0c0fe56c048 /]# mkdir -p /data/gpconfigs

[root@b0c0fe56c048 /]# chown -Rf gpadmin:gpadmin /data/

2.3.5 切换gpadmin用户

[root@b0c0fe56c048 /]# su gpadmin

2.3.6 查看gpadmin用户环境变量

在~/.bashrc文件中添加source /usr/local/greenplum-db/greenplum_path.sh

这样在root用户下修改了环境后，一旦切换到gpadmin用户，会自动加载。

后续还有其他变量需要添加时，也添加在该文件中

[gpadmin@b0c0fe56c048 /]$ cat ~/.bashrc

# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
        . /etc/bashrc
fi

# Uncomment the following line if you don't like systemctl's auto-paging feature:
# export SYSTEMD_PAGER=

# User specific aliases and functions
source /usr/local/greenplum-db/greenplum_path.sh

刚修改好，可以手动source一下。source ~/.bashrc

2.3.7 SSH互信设置

[gpadmin@b0c0fe56c048 /]$ gpssh-exkeys -h b0c0fe56c048
[STEP 1 of 5] create local ID and authorize on local host

[STEP 2 of 5] keyscan all hosts and update known_hosts file

[STEP 3 of 5] retrieving credentials from remote hosts

[STEP 4 of 5] determine common authentication file content

[STEP 5 of 5] copy authentication files to all remote hosts

[INFO] completed successfully

2.3.8 配置初始化文件

2.3.8.1 拷贝文件到配置目录

[gpadmin@b0c0fe56c048 /]$ cp /usr/local/greenplum-db/docs/cli_help/gpconfigs/gpinitsystem_config /data/gpconfigs/

2.3.8.2 创建host文件

[gpadmin@b0c0fe56c048 /]$ touch /data/gpconfigs/hostfile
[gpadmin@b0c0fe56c048 /]$ echo "b0c0fe56c048" > /data/gpconfigs/hostfile

2.3.8.3 修改初始化文件

主要关注master和segment的目录，主机名和数据库端口

[gpadmin@b0c0fe56c048 /]$ vi /data/gpconfigs/gpinitsystem_config

[gpadmin@b0c0fe56c048 /]$ cat /data/gpconfigs/gpinitsystem_config

# FILE NAME: gpinitsystem_config

# Configuration file needed by the gpinitsystem

################################################
#### REQUIRED PARAMETERS
################################################

#### Name of this Greenplum system enclosed in quotes.
ARRAY_NAME="Greenplum Data Platform"

#### Naming convention for utility-generated data directories.
SEG_PREFIX=gpseg

#### Base number by which primary segment port numbers
#### are calculated.
PORT_BASE=6000

#### File system location(s) where primary segment data directories
#### will be created. The number of locations in the list dictate
#### the number of primary segments that will get created per
#### physical host (if multiple addresses for a host are listed in
#### the hostfile, the number of segments will be spread evenly across
#### the specified interface addresses).
declare -a DATA_DIRECTORY=(/data/gpdata/primary /data/gpdata/primary)

#### OS-configured hostname or IP address of the master host.
MASTER_HOSTNAME=b0c0fe56c048

#### File system location where the master data directory
#### will be created.
MASTER_DIRECTORY=/data/gpdata/master

#### Port number for the master instance.
MASTER_PORT=5432

#### Shell utility used to connect to remote hosts.
TRUSTED_SHELL=ssh

#### Maximum log file segments between automatic WAL checkpoints.
CHECK_POINT_SEGMENTS=8

#### Default server-side character set encoding.
ENCODING=UNICODE

################################################
#### OPTIONAL MIRROR PARAMETERS
################################################

#### Base number by which mirror segment port numbers
#### are calculated.
#MIRROR_PORT_BASE=7000

#### File system location(s) where mirror segment data directories
#### will be created. The number of mirror locations must equal the
#### number of primary locations as specified in the
#### DATA_DIRECTORY parameter.
#declare -a MIRROR_DATA_DIRECTORY=(/data1/mirror /data1/mirror /data1/mirror /data2/mirror /data2/mirror /data2/mirror)


################################################
#### OTHER OPTIONAL PARAMETERS
################################################

#### Create a database of this name after initialization.
#DATABASE_NAME=name_of_database

#### Specify the location of the host address file here instead of
#### with the the -h option of gpinitsystem.
#MACHINE_LIST_FILE=/home/gpadmin/gpconfigs/hostfile_gpinitsystem

2.3.9 初始化数据库

[gpadmin@b0c0fe56c048 /]$ gpinitsystem -c /data/gpconfigs/gpinitsystem_config -h /data/gpconfigs/hostfile

2.3.10 连接数据库

[gpadmin@b0c0fe56c048 /]$ psql -p 5432 -d postgres
psql (9.4.24)
Type "help" for help.

2.3.11 修改密码

[gpadmin@b0c0fe56c048 /]$ psql -p 5432 -d postgres
psql (9.4.24)
Type "help" for help.

postgres=# \password gpadmin
Enter new password:
Enter it again:

三、远程连接

3.1 开放端口

通过指令查询任务进程，以及端口。

容器的6000,6001,5432，分别对应服务器的6000,6001,5432端口，主要开放5432端口允许外部访问即可

[gpadmin@b0c0fe56c048 /]$ ps -ef | grep greenplum
gpadmin   3560     1  0 02:39 ?        00:00:00 /usr/local/greenplum-db-6.4.0/bin/postgres -D /data/gpdata/primary/gpseg0 -p 6000
gpadmin   3561     1  0 02:39 ?        00:00:00 /usr/local/greenplum-db-6.4.0/bin/postgres -D /data/gpdata/primary/gpseg1 -p 6001
gpadmin   3582     0  0 02:39 ?        00:00:00 /usr/local/greenplum-db-6.4.0/bin/postgres -D /data/gpdata/master/gpseg-1 -p 5432 -E
gpadmin   3725   405  0 02:43 pts/1    00:00:00 grep --color=auto greenplum

3.2 修改pg_hba.conf

#添加如下一行即可，所有用户，所有IP均可访问
#生产环境中可自行选择配置
[gpadmin@b0c0fe56c048 /]$ vi /data/gpdata/master/gpseg-1/pg_hba.conf

host     all         all         all       trust

3.3 重启greenplum

3.3.1 gpstop关闭失败

20230322:02:45:30:003728 gpstop:b0c0fe56c048:gpadmin-[CRITICAL]:-gpstop failed. (Reason='Environment Variable MASTER_DATA_DIRECTORY not set!') exiting...

在使用gpstop的时候，提示关闭失败，提示你设置MASTER_DATA_DIRECTORY

[gpadmin@b0c0fe56c048 /]$ gpstop
20230322:02:45:30:003728 gpstop:b0c0fe56c048:gpadmin-[INFO]:-Starting gpstop with args:
20230322:02:45:30:003728 gpstop:b0c0fe56c048:gpadmin-[INFO]:-Gathering information and validating the environment...
20230322:02:45:30:003728 gpstop:b0c0fe56c048:gpadmin-[CRITICAL]:-gpstop failed. (Reason='Environment Variable MASTER_DATA_DIRECTORY not set!') exiting...

3.3.2 设置MASTER_DATA_DIRECTORY

在~/.bashrc文件中添加如下两行记录即可

[gpadmin@b0c0fe56c048 /]$ vi ~/.bashrc

MASTER_DATA_DIRECTORY=/data/gpdata/master/gpseg-1
export MASTER_DATA_DIRECTORY

[gpadmin@b0c0fe56c048 /]$ source ~/.bashrc

3.4 关闭gp

[gpadmin@b0c0fe56c048 /]$ gpstop
.....
20230322:02:46:57:003750 gpstop:b0c0fe56c048:gpadmin-[INFO]:-Cleaning up leftover shared memory

3.5 启动gp

[gpadmin@b0c0fe56c048 /]$ gpstart
......
20230322:02:47:21:003933 gpstart:b0c0fe56c048:gpadmin-[INFO]:-Database successfully started

3.6 远程连接

Dbeaver使用GreenPlum驱动连接数据库