centos6.4安装bert-as-service

版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/liuxiao723846/article/details/84954580

BERT是由Google开发的用于预训练语言表示的NLP模型。它利用网上公开提供的大量纯文本数据,并以无人监督的方式进行培训。预训练BERT模型对于每种语言来说都是相当昂贵但一次性的过程。幸运的是,Google发布了几个预先训练好的模型,您可以从这里下载(项目git地址:https://github.com/hanxiao/bert-as-service)。

就像官网介绍的一样,bert使用起来非常简单,只需要两行代码。但是安装过程需要升级python、升级gcc(会重新编译linux内核),导致绝大部分初学者望而却步。今天,我们就详细介绍如何在centos6.4上徒手安装bert-server。

一、安装bert-server

官网介绍,bert服务至少需要python3.5和tensorflow1.11。所以,我们首要任务升级python。查看版本信息:

# cat /etc/redhat-release 
CentOS release 6.4 (Final)

$ python -V
Python 2.6.6

1、升级python:

https://blog.csdn.net/liuxiao723846/article/details/84946353

2、python3上安装pip:

https://blog.csdn.net/liuxiao723846/article/details/84947609

3、pip安装bert:

接下来,我们按照官网中通过pip安装bert:

# python -m pip install bert-serving-server

执行后半段会报如下错误:

提示需要升级pip。

4、升级pip:

升级后,再次安装bert-server

二、启动bert-server

安装成功后,根据官网说明我们可以启动使用bert-server。

首先,下载bert模型:

cd /data
wget https://storage.googleapis.com/bert_models/2018_11_03/chinese_L-12_H-768_A-12.zip
unzip chinese_L-12_H-768_A-12.zip

接下来,通过如下命令启动bert-server,但是发现没有bert-serving-start 这个命令,于是搜一下...

启动

发现会报没有tensorflow错误,于是开始安装tf。

1、安装tf:

根据官网说明, Tensorflow >= 1.10,所以:

python -m pip install tensorflow==1.10

easy,一次性搞定。。。满怀欣喜开始启动bert-server。

噩耗刚刚开始,这个错误是gcc版本不对导致。解决方法如下:

https://blog.csdn.net/liuxiao723846/article/details/84955136

解决完/usr/lib64/libstdc++.so.6问题后,继续启动bert-server,又出现了如下错误:

根据上面经验,GLIBC_2.16问题还是内核版本不支持,解决方法:

https://blog.csdn.net/liuxiao723846/article/details/84959784

至此,没有其他坑了。

2、启动:

三、bert-client安装:

pip install bert-serving-client

官网说可以在python2上运行,但实际上可能也需要python3的环境。

最后,客户端和服务端要配套才能连上,指定服务端版本:

$ python -m pip install bert-serving-server==1.5.0
Collecting bert-serving-server==1.5.0
  Downloading https://files.pythonhosted.org/packages/32/e7/ada782b2a8758cb70feed133ec7be64100d8997a43f320951dcc830a049e/bert_serving_server-1.5.0-py3-none-any.whl (43kB)
    100% |████████████████████████████████| 51kB 35kB/s 
Requirement already satisfied (use --upgrade to upgrade): pyzmq>=17.1.0 in /usr/local/python352/lib/python3.5/site-packages (from bert-serving-server==1.5.0)
Requirement already satisfied (use --upgrade to upgrade): numpy in /usr/local/python352/lib/python3.5/site-packages (from bert-serving-server==1.5.0)
Requirement already satisfied (use --upgrade to upgrade): termcolor>=1.1 in /usr/local/python352/lib/python3.5/site-packages (from bert-serving-server==1.5.0)
Requirement already satisfied (use --upgrade to upgrade): GPUtil>=1.3.0 in /usr/local/python352/lib/python3.5/site-packages (from bert-serving-server==1.5.0)
Requirement already satisfied (use --upgrade to upgrade): six in /usr/local/python352/lib/python3.5/site-packages (from bert-serving-server==1.5.0)
Installing collected packages: bert-serving-server
  Found existing installation: bert-serving-server 1.5.1
    Uninstalling bert-serving-server-1.5.1:
      Successfully uninstalled bert-serving-server-1.5.1
Successfully installed bert-serving-server-1.5.0
You are using pip version 8.1.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

猜你喜欢

转载自blog.csdn.net/liuxiao723846/article/details/84954580