barman备份软件离线安装渡劫文

一、前言

Barman是postgreSQL数据库的一款比较好用的备份软件,目前网上很多安装部署都是以Online的模式进行安装部署的,其相关的package也可以被自动的安装部署,所以相对简单。但是,在离线模式下,安装却有很多"坑",今天总结的这篇文章,可以帮助离线安装的小伙伴们渡劫。

二、环境描述

基础环境:redhat-7.5 x86_64
软件版本:barman-2.12

官方在说明文档描述,部署barman-2.12的时候对环境有以下要求:

• Linux/Unix

• Python >= 3.4

• Python modules:
– argcomplete
– argh >= 0.21.2
– psycopg2 >= 2.4.2
– python-dateutil
– setuptools
- six

• PostgreSQL >= 8.3

• rsync >= 3.0.4 (optional for PostgreSQL >= 9.2)

三、实际部署安装

在安装中,我使用的是当前最新版本:python3.9,以及其他python模块的最新版本,具体如下:

[root@barman soft]# ls -l
total 53404
-rw-r--r--.  1 root root    38214 Nov 23 15:32 argcomplete-1.12.1-py2.py3-none-any.whl
-rw-r--r--.  1 root root    30872 Nov 23 15:35 argh-0.26.2-py2.py3-none-any.whl
-rw-r--r--.  1 root root  1262415 Nov 13 16:48 barman-2.12.tar.gz
-rw-r--r--.  1 root root    22000 Nov 24 17:01 _bz2.cpython-36m-x86_64-linux-gnu.so
-rw-r--r--.  1 root root 25915738 Jun 25  2019 postgresql-11.4.tar.gz
-rw-r--r--.  1 root root   427474 Nov 23 23:24 psycopg2-2.8.6.tar.gz
drwxr-xr-x. 18 1000 1000     4096 Nov 24 21:59 Python-3.9.0
-rw-r--r--.  1 root root 26724009 Nov 17 11:56 Python-3.9.0.tgz
-rw-r--r--.  1 root root   227183 Nov 23 15:38 python_dateutil-2.8.1-py2.py3-none-any.whl
-rw-r--r--.  1 root root    10938 Nov 23 22:23 six-1.14.0-py2.py3-none-any.whl

1)安装常用的包和工具

yum -y install binutils gcc gcc-c++ glibc glibc-common  glibc-devel ksh libaio libaio-devel libgcc libstdc++ libstdc++-devel make sysstat openssh-clients compat-libcap1   xorg-x11-utils xorg-x11-xauth elfutils unixODBC unixODBC-devel libXp elfutils-libelf elfutils-libelf-devel smartmontools

2)安装rsync

yum install rsync -y

rpm -qa rsync
rsync-3.1.2-4.el7.x86_64

3)安装python3

tar -xvzf Python-3.8.6.tgz
cd Python-3.8.6

./configure
make && make install
rm -rf /usr/bin/python
ln -s /usr/bin/python /usr/local/bin/python3
或者:
./configure --prefix /python3
make && make install
rm -rf /usr/bin/python
rm -rf /usr/bin/pip
ln -s /python3/bin/python3 /usr/bin/python 
ln -s /python3/bin/pip3 /usr/bin/pip 

vi /etc/profile
export PATH=$PATH:/python3/bin

注:
需要注意的是,在redhat- 7.5下,yum的执行是通过python2.7来解释的的,由于我们用软连接变更了/usr/bin/python的定义,所以我们可能会无法使用yum:

[root@barman Python-3.9.0]# yum list
  File "/usr/bin/yum", line 30
    except KeyboardInterrupt, e:
                            ^
SyntaxError: invalid syntax

此时,我们只需要在/usr/bin/yum命令中改变解释器为2.7即可:

#!/usr/bin/python2.7
import sys
try:
    import yum
except ImportError:
    print >> sys.stderr, """\
There was a problem importing one of the Python modules
required to run yum. The error leading to this problem was:

   %s

Please install a package which provides this module, or
verify that the module is installed correctly.

It's possible that the above module doesn't match the
current version of Python, which is:
%s

If you cannot solve this problem yourself, please go to
the yum faq at:
  http://yum.baseurl.org/wiki/Faq

""" % (sys.exc_value, sys.version)
    sys.exit(1)

sys.path.insert(0, '/usr/share/yum-cli')
try:
    import yummain
    yummain.user_main(sys.argv[1:], exit_code=True)
except KeyboardInterrupt, e:
    print >> sys.stderr, "\n\nExiting on user cancel."
    sys.exit(1)

python安装好后,会默认安装好pip和setuptools,所以不用另外安装,但是python的其他控件需要另外安装:

4)安装argcomplete:

[root@barman soft]# pip install argcomplete-1.12.1-py2.py3-none-any.whl

5)安装argh:

[root@barman soft]# pip install argh-0.26.2-py2.py3-none-any.whl

6)安装six:

[root@barman soft]# pip install six-1.14.0-py2.py3-none-any.whl 

7)安装python-dateutil:

[root@barman soft]# pip install python_dateutil-2.8.1-py2.py3-none-any.whl

注:在安装python-dateutil前必须先安装six模块,否则会报错:

WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Processing ./python_dateutil-2.8.1-py2.py3-none-any.whl
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/six/

8)安装psycopg2-2.8.6

[root@barman soft]# tar -xvzf psycopg2-2.8.6.tar.gz
[root@barman psycopg2-2.8.6]# cd psycopg2-2.8.6
[root@barman psycopg2-2.8.6]# python setup.py build

可以发现报了以下错误:

Error: pg_config executable not found.

pg_config is required to build psycopg2 from source.  Please add the directory
containing pg_config to the $PATH or specify the full executable path with the
option:

    python setup.py build_ext --pg-config /path/to/pg_config build ...

or with the pg_config option in 'setup.cfg'.

If you prefer to avoid building psycopg2 from source, please install the PyPI
'psycopg2-binary' package instead.

For further information please check the 'doc/src/install.rst' file (also at
<https://www.psycopg.org/docs/install.html>).

这是因为没有找到pg_config文件,而它是在postgresql的服务器端或者客户端中所包含的,所以我们要先安装postgresql的服务端或客户端。

* 安装postgreSQL客户端:

mkdir -p /pg_client
tar -xvzf postgresql-11.4.tar.gz

yum install -y readline*
yum install -y zlib*
yum install -y gcc*
yum install -y perl*
yum install -y tcl*

cd postgresql-11.4
./configure --prefix=/pg_client --with-python --with-perl
make
make world
make -C src/bin install
make -C src/include install
make -C src/interfaces install
make -C doc install

vi /etc/profile
MANPATH=/pg_client/share/man:$MANPATH
LD_LIBRARY_PATH=/pg_client/lib
PG_HOME=/pg_client
PATH=$PATH:$HOME/bin:$PG_HOME/bin
export PATH PG_HOME LD_LIBRARY_PATH MANPATH

再次编译安装psycopg2-2.8.6

[root@barman psycopg2-2.8.6]# cd psycopg2-2.8.6
[root@barman psycopg2-2.8.6]# python setup.py build
[root@barman psycopg2-2.8.6]# python setup.py install

9)查看安装的python包:

[root@barman psycopg2-2.8.6]# pip list
Package         Version
--------------- -------
argcomplete     1.12.1
argh            0.26.2
pip             20.2.1
psycopg2        2.8.6
python-dateutil 2.8.1
setuptools      49.2.1
six             1.14.0

其实,可以看到,部署barman需要的相关Python模块有以上这些部分(有些是模块本身需要依赖的模块)。

10)安装barman

tar -xvzf barman-2.12.tar.gz 
chown -R postgres:postgres barman-2.12

cd barman-2.12
python setup.py build
python setup.py install

* 验证barman:

[root@barman barman-2.12]# barman -h

发现报了如下错误:

Traceback (most recent call last):
  File "/python3/bin/barman", line 33, in <module>
    sys.exit(load_entry_point('barman==2.12', 'console_scripts', 'barman')())
  File "/python3/bin/barman", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/python3/lib/python3.9/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/python3/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/python3/lib/python3.9/site-packages/barman-2.12-py3.9.egg/barman/cli.py", line 32, in <module>
    import barman.config
  File "/python3/lib/python3.9/site-packages/barman-2.12-py3.9.egg/barman/config.py", line 32, in <module>
    from barman import output
  File "/python3/lib/python3.9/site-packages/barman-2.12-py3.9.egg/barman/output.py", line 30, in <module>
    from barman.infofile import BackupInfo
  File "/python3/lib/python3.9/site-packages/barman-2.12-py3.9.egg/barman/infofile.py", line 27, in <module>
    import barman.compression
  File "/python3/lib/python3.9/site-packages/barman-2.12-py3.9.egg/barman/compression.py", line 22, in <module>
    import bz2
  File "/python3/lib/python3.9/bz2.py", line 18, in <module>
    from _bz2 import BZ2Compressor, BZ2Decompressor
ModuleNotFoundError: No module named '_bz2'

这是由于python缺少了_bz2.cpython-36m-x86_64-linux-gnu.so文件,解决该问题的处理方式是:
下载这个文件,或者在python3.6的安装版本中可以找到,其路径位置是:/python3/lib/python3.6/lib-dynload,也可以直接从网上下载:
链接:https://pan.baidu.com/s/1rPd7KSe6rpxxVVW2Bga-Vg
提取码:yptk

然后改名,并复制到/python3/lib/python3.9/lib-dynload下

需要注意:

这里改名需要注意,在网上查到的方式是将_bz2.cpython-36m-x86_64-linux-gnu.so中的36m改成对应版本的规则,比如python3.7,就改成_bz2.cpython-36m-x86_64-linux-gnu.so
但是在python 3.8和python3.9中,改成38m和39m是无效的;
我们最好是到/python3/lib/python3.9/lib-dynload目录下,看一下其他文件命名规则:

......
-rwxr-xr-x. 1 root root   65768 Nov 24 21:58 _testmultiphase.cpython-39-x86_64-linux-gnu.so
-rwxr-xr-x. 1 root root 1198000 Nov 24 21:58 unicodedata.cpython-39-x86_64-linux-gnu.so
-rwxr-xr-x. 1 root root   27288 Nov 24 21:59 xxlimited.cpython-39-x86_64-linux-gnu.so
-rwxr-xr-x. 1 root root  144472 Nov 24 21:58 _xxsubinterpreters.cpython-39-x86_64-linux-gnu.so
-rwxr-xr-x. 1 root root   66104 Nov 24 21:58 _xxtestfuzz.cpython-39-x86_64-linux-gnu.so
-rwxr-xr-x. 1 root root  160336 Nov 24 21:58 zlib.cpython-39-x86_64-linux-gnu.so
-rwxr-xr-x. 1 root root  135440 Nov 24 21:58 _zoneinfo.cpython-39-x86_64-linux-gnu.so
......

可以发现这里的版本都不是39m了,直接是39
所以对于该文件,我们要改名为:_bz2.cpython-39-x86_64-linux-gnu.so

[root@barman lib-dynload]# cp /home/soft/_bz2.cpython-36m-x86_64-linux-gnu.so ./_bz2.cpython-39-x86_64-linux-gnu.so

加执行权限:

[root@barman lib-dynload]# chmod +x _bz2.cpython-39-x86_64-linux-gnu.so

再次执行barman,会发现报如下错误:

ImportError: libbz2.so.1.0: cannot open shared object file: No such file or directory

* 安装bzip2

[root@barman lib-dynload]# yum install -y bzip2* 
[root@barman lib-dynload]# cd /usr/lib64
[root@barman lib64]# ls -l libbz2.so.1.0.6
-rwxr-xr-x. 1 root root 68192 Jul 31  2014 libbz2.so.1.0.6

创建连接:

[root@barman lib64]# ln -s libbz2.so.1.0.6 libbz2.so.1.0

再次测试验证:

[root@barman lib64]# barman -h
usage: barman [-h] [-v] [-c CONFIG] [--color {never,always,auto}] [--log-level {NOTSET,DEBUG,INFO,WARNING,ERROR,CRITICAL}] [-q] [-d] [-f {console,json}]
              {archive-wal,backup,check,check-backup,cron,delete,diagnose,get-wal,list-backup,list-files,list-server,put-wal,rebuild-xlogdb,receive-wal,recover,show-backup,show-server,replication-status,status,switch-wal,switch-xlog,sync-info,sync-backup,sync-wals}
              ...

positional arguments:
  {archive-wal,backup,check,check-backup,cron,delete,diagnose,get-wal,list-backup,list-files,list-server,put-wal,rebuild-xlogdb,receive-wal,recover,show-backup,show-server,replication-status,status,switch-wal,switch-xlog,sync-info,sync-backup,sync-wals}
    archive-wal         Execute maintenance operations on WAL files for a given server. This command processes any incoming WAL files for the server and archives
                        them along the catalogue.
    backup              Perform a full backup for the given server (supports 'all')
    check               Check if the server configuration is working. This command returns success if every checks pass, or failure if any of these fails
    check-backup        Make sure that all the required WAL files to check the consistency of a physical backup (that is, from the beginning to the end of the full
                        backup) are correctly archived. This command is automatically invoked by the cron command and at the end of every backup operation.
    cron                Run maintenance tasks (global command)
    delete              Delete a backup
    diagnose            Diagnostic command (for support and problems detection purpose)
    get-wal             Retrieve WAL_NAME file from SERVER_NAME archive. The content will be streamed on standard output unless the --output-directory option is
                        specified.
    list-backup         List available backups for the given server (supports 'all')
    list-files          List all the files for a single backup
    list-server         List available servers, with useful information
    put-wal             Receive a WAL file from SERVER_NAME and securely store it in the incoming directory. The file will be read from standard input in tar format.
    rebuild-xlogdb      Rebuild the WAL file database guessing it from the disk content.
    receive-wal         Start a receive-wal process. The process uses the streaming protocol to receive WAL files from the PostgreSQL server.
    recover             Recover a server at a given time, name, LSN or xid
    show-backup         This method shows a single backup information
    show-server         Show all configuration parameters for the specified servers
    replication-status  Shows live information and status of any streaming client
    status              Shows live information and status of the PostgreSQL server
    switch-wal          Execute the switch-wal command on the target server
    switch-xlog         Execute the switch-wal command on the target server
    sync-info           Output the internal synchronisation status. Used to sync_backup with a passive node
    sync-backup         Command that synchronises a backup from a master to a passive node
    sync-wals           Command that synchronises WAL files from a master to a passive node

optional arguments:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit
  -c CONFIG, --config CONFIG
                        uses a configuration file (defaults: ~/.barman.conf, /etc/barman.conf, /etc/barman/barman.conf)
  --color {never,always,auto}, --colour {never,always,auto}
                        Whether to use colors in the output (default: 'auto')
  --log-level {NOTSET,DEBUG,INFO,WARNING,ERROR,CRITICAL}
                        Override the default log level
  -q, --quiet           be quiet (default: False)
  -d, --debug           debug output (default: False)
  -f {console,json}, --format {console,json}
                        output format (default: 'console')

Barman by 2ndQuadrant (www.2ndQuadrant.com)

可以发现barman可以正常运行,安装成功!

11)复制配置文件

将配置复制到/etc下,安装全部完成:

cp -r /home/soft/barman-2.12/doc/barman.conf /etc/barman.conf
cp -r /home/soft/barman-2.12/doc/barman.d /etc/barman.d

猜你喜欢

转载自blog.csdn.net/wx370092877/article/details/110100383