Speaking from a few days ago to start learning to use bert, first of all there are two, one is bert google open source original, one is pytorch version, here we are introduced bert original, providing greater control over the original sex, if you want to save can directly use the second.
A, python requirements
It should be noted that tensorflow-gpu currently only supports 2.7,3.3-3.6 , if you are the wrong version of python, self-reload (or use the following virtual environments).
Second, the installation Anaconda3 + tensorflow-gpu
Install this stuff on their own at random, I installed for faster installation GPU-tensorflow . pip install it from outside the network is installed, not that slow, easier to hang up, so the use of this, the installation process itself Baidu.
First, create a custom environment python version (version limit above)
Open Anaconda Prompt , Anaconda position at the beginning of the file folder.
conda create -n learn python=3.6
learn:环境名字，自定义
python=3.6:python版本号Second, the installation tensorflow-gpu
Here I would like to remind, the latest version of bert need tensorflow-GPU> 1.10.0 , otherwise they will be reported the following error
AttributeError: module 'tensorflow.contrib.tpu' has no attribute 'InputPipelineConfig'There can not be directly PIP , as it will directly install the latest reference: https://www.tensorflow.org/install/source_windows , reference can not turn wall:
====接上面创建完环境===
activate learn 激活环境
pip install tensorflow-gpu==1.10.0 安装Can not be installed directly, you need to install cuDNN, CUDA
Third, the installation cuDNN + CUDA
Two versions of the above reference, here is CUDA9 + cuDNN7
First, install CUDA9
Address: https://developer.nvidia.com/cuda-toolkit-archive
After selecting local, in fact, not a big difference between online installation, an offline installation.
The default installation process is not to say, if you select a custom, three remember not to put a file folder! ! ! Be sure to distinguish it! ! !
This is my custom file folders.
Second, the installation cuDNN7
Address: https://developer.nvidia.com/cudnn , this need to log in, but you can use a micro letter, QQ need to verify with the mailbox, but you are not receiving e-mail.
Be sure to note the version! ! ! And then select window10, on it.
Downloaded is a compressed package here is not cudnn 9 version, but the version of cuda, behind v7 is , which is:
In front of high energy! ! !
Here are the first three files do not move, before you install CUDA three folders If the default is
If the custom is this: the name of a folder here my custom , custom must distinguish, Sample, Documentation and the last one, in order to distinguish.
The three compressed files under the folder cuDNN (one each for each folder) is placed inside the three Toolkit folder corresponding to the file, and Sample Documentation not remember, this is the compressed file, not folders , copied to the corresponding directory.
Third, define the environment variable
The following environment variables into a line to the path inside.
E:\GPU\NVIDIA GPU Computing Toolkit\bin\cudart64_90.dll;
E:\GPU\NVIDIA GPU Computing Toolkit\lib\x64;
E:\GPU\NVIDIA GPU Computing Toolkit\include;
E:\GPU\NVIDIA GPU Computing Toolkit\extras\CUPTI\libx64;
E:\GPU\NVIDIA Corporation Samples\bin\win64;
E:\GPU\NVIDIA Corporation Samples\common\lib\x64;
E:\GPU\NVIDIA GPU Computing Toolkit\bin;
E:\GPU\NVIDIA GPU Computing Toolkit\libnvvp;This time you will find still can not run tensorflow-gpu, this time you save everything, restart the computer, you can use. NAVIDA need to re-identify.
Fourth, the pit!
Bert run you will find INFO: tensorflow: Running train on CPU , FML, the magic of it, in fact, does not matter, it just means not on the TPU, but includes CPU and GPU, then you will find:
name: GeForce GTX 960M major: 5 minor: 0 memoryClockRate(GHz): 1.176
pciBusID: 0000:01:00.0
totalMemory: 4.00GiB freeMemory: 3.35GiBThis is not to go on the GPU yet, which makes me a lot of heavy equipment.
