我的电脑配置比较垃圾:3代i5+GT750m
装过不下50次caffe和ubuntu的我建议先装tensorflow-gpu(cuda9.1不支持tensorflow-gpu)再来caffe-gpu
ubuntu16.04自带python2.7和3.5
tensorflow-gpu安装非常简单
pip install tensorflow-gpu == 1.x.x #也可以前面加sudo,后面==是要安装的版本,按错了直接换就行
pip install tensorflow == 1.x.x #这是cpu的
配套tensorflow的肯定就是Keras和theano
pip install keras == 2.1.x pip install theano
注意:有些显卡驱动版本不兼容,必须要配套的tensorflow,所以省时间报错就换版本 我的是tensorflow1.2 keras2.1.3
CPU的随便按 pip install tensorflow keras 这样可以安装俩的最新版
当然,python里面一堆乱七八糟的包要按好,比如numpy之类的。。。
而且ubuntu17.04安装caffe可以像上面按tensorflow一样。
一、安装nVidia驱动,配置cuda,cudnn运行环境
1.安装相关依赖项
sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev> libopencv-dev libhdf5-serial-dev protobuf-compiler sudo apt-get install --no-install-recommends libboost-all-dev sudo apt-get install libopenblas-dev liblapack-dev libatlas-base-dev sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev
2.安装NVIDIA驱动
驱动最简单的方法是在Ubuntu系统自带的设置里更改,去官网下载的驱动会比较崩还麻烦。
3.安装CUDA
首先在官网上( https://developer.nvidia.com/cuda-downloads )下载CUDA,选择自己合适的版本。该链接界面只显示最新版本。若需要下载以前的版本,可在页面下方点击Legacy Releases,选择自己需要的其他版本。
下载完成后执行以下命令:
sudo chmod 777 cuda_8.0.44_linux.run sudo ./cuda_8.0.44_linux.run
安装时首先会有一系列提示让你确认到100%,但是注意,有个让你选择是否安装nvidia361驱动时,一定要选择否:
Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 361.62?
这个东西是CUDA自带的驱动,可能和你的显卡不兼容。上面装驱动的方法是我觉得最简单不烦人的方法。
安装cuda时可能有下面的信息
Installing the CUDA Toolkit in /usr/local/cuda-8.0 … Missing recommended library: libGLU.so Missing recommended library: libX11.so Missing recommended library: libXi.so Missing recommended library: libXmu.so
原因是缺少相关的依赖库,安装相应库就解决了:
sudo apt-get install freeglut3-dev build-essential libx11-dev libxmu-dev libxi-dev libgl1-mesa-glx libglu1-mesa libglu1-mesa-dev
环境变量配置:
打开~/.bashrc文件: sudo gedit ~/.bashrc
将以下内容写入到~/.bashrc尾部:
export PATH=/usr/local/cuda-8.0/bin${PATH:+:${PATH}} export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
测试CUDA的samples:
cd /usr/local/cuda-8.0/samples/1_Utilities/deviceQuery make sudo ./deviceQuery
如果显示一些乱七八糟的关于GPU的信息,则说明安装成功。
4.配置cuDNN
cuDNN是GPU加速计算深层神经网络的库。 首先去官网https://developer.nvidia.com/rdp/cudnn-download 下载cuDNN,需要注册一个账号才能下载。
下载cuDNN5.1 之后进行解压:
cd cuda/include sudo cp cudnn.h /usr/local/cuda/include
再进入lib64目录下的动态文件进行复制和链接:
cd .. cd lib64 sudo cp lib* /usr/local/cuda/lib64/
cd /usr/local/cuda/lib64/ sudo rm -rf libcudnn.so libcudnn.so.5
sudo ln -s libcudnn.so.5.1.10 libcudnn.so.5 #生成软衔接(注意这里要和自己下载的cudnn版本对应,可以在/usr/local/cuda/lib64下查看自己libcudnn的版本) sudo ln -s libcudnn.so.5 libcudnn.so #生成软链接执行
sudo ldconfig -v #必须在/usr/local/cuda/lib64/目录下执行,否则可能会报libcudnn.so.5: cannot open shared object file: No such file or directory
或者
sudo ldconfig /usr/local/cuda/lib64 #cudnn就配置完成了。
三、配置caffe
(1)将终端cd到要安装caffe的位置。
(2)从github上获取caffe:
git clone https://github.com/BVLC/caffe.git
注意:若没有安装Git,需要先安装Git:
sudo apt-get install git
(3)因为make指令只能make Makefile.config文件,而Makefile.config.example是caffe给出的makefile例子,因此,首先将Makefile.config.example的内容复制到Makefile.config:
sudo cp Makefile.config.example Makefile.config
(4)打开并修改配置文件:
sudo gedit Makefile.config #打开Makefile.config文件
根据个人情况修改文件:
a.若使用cudnn,则
将
#USE_CUDNN := 1
修改成:
USE_CUDNN := 1
b.若使用的opencv版本是3的,则
将
#OPENCV_VERSION := 3
修改为:
OPENCV_VERSION := 3
c.若要使用python来编写layer,则
将
#WITH_PYTHON_LAYER := 1
修改为
WITH_PYTHON_LAYER := 1
d. 重要的一项 :
将# Whatever else you find you need goes here.下面的
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib
修改为:
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/hdf5/serial
这是因为ubuntu16.04的文件包含位置发生了变化,尤其是需要用到的hdf5的位置,所以需要更改这一路径。
这里贴出我修改后的Makefile.config文件:
## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!
# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1 #使用cuDNN
# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1
# uncomment to disable IO dependencies and corresponding data layers
# USE_OPENCV := 0
# USE_LEVELDB := 0
# USE_LMDB := 0
# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
# You should not set this flag if you will be reading LMDBs with any
# possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1
# Uncomment if you're using OpenCV 3
OPENCV_VERSION := 3 #使用openCV3
# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++
# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda #CUDA路径
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr
# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility.
# For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility.
# For CUDA >= 9.0, comment the *_20 and *_21 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
-gencode arch=compute_20,code=sm_21 \
-gencode arch=compute_30,code=sm_30 \
-gencode arch=compute_35,code=sm_35 \
-gencode arch=compute_50,code=sm_50 \
-gencode arch=compute_52,code=sm_52 \
-gencode arch=compute_60,code=sm_60 \
-gencode arch=compute_61,code=sm_61 \
-gencode arch=compute_61,code=compute_61 #计算能力
# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas #使用哪种BLAS:atlas or mkl or openblas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
# BLAS_INCLUDE := /path/to/your/blas
# BLAS_LIB := /path/to/your/blas
# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib
# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app
# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
# PYTHON_INCLUDE := /usr/include/python2.7 \
# /usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
ANACONDA_HOME := $(HOME)/anaconda2 #使用anaconda,anaconda的路径
PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
$(ANACONDA_HOME)/include/python2.7 \
$(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include #anaconda的python头文件路径
# Uncomment to use Python 3 (default is Python 2)
# PYTHON_LIBRARIES := boost_python3 python3.5m
# PYTHON_INCLUDE := /usr/include/python3.5m \
# /usr/lib/python3.5/dist-packages/numpy/core/include
# We need to be able to find libpythonX.X.so or .dylib.
# PYTHON_LIB := /usr/lib
PYTHON_LIB := $(ANACONDA_HOME)/lib #anaconda的python库目录
# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib
# Uncomment to support layers written in Python (will link against Python libs)
WITH_PYTHON_LAYER := 1 #使用python层
# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial #头文件路径
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/hdf5/serial #库路径
# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib
# NCCL acceleration switch (uncomment to build with NCCL)
# https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0)
# USE_NCCL := 1
# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1
# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute
# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1
# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0 #GPU编号
# enable pretty build (comment to see full commands)
Q ?= @
(5)修改makefile文件
打开makefile文件,做如下修改:
将:
NVCCFLAGS +=-ccbin=$(CXX) -Xcompiler-fPIC $(COMMON_FLAGS)
替换为:
NVCCFLAGS += -D_FORCE_INLINES -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)
(6)编辑/usr/local/cuda/include/host_config.h
将其中的第115行注释掉:
将
#error-- unsupported GNU version! gcc versions later than 4.9 are not supported!
改为
//#error-- unsupported GNU version! gcc versions later than 4.9 are not supported!
(7)编译
make all -j4 #-j根据自己电脑配置决定 -j2代表2线程 4为4线程 现在8代i7可以12线程
出错就make clean 重来
编译过程中可能会出现如下错误:
错误内容1:
"fatal error: hdf5.h: 没有那个文件或目录"
解决办法:
step1:在Makefile.config文件的第85行,添加/usr/include/hdf5/serial/ 到 INCLUDE_DIRS,也就是把下面第一行代码改为第二行代码。
将:
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
替换为:
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/
stept2:在Makefile文件的第173行,把 hdf5_hl 和hdf5修改为hdf5_serial_hl 和 hdf5_serial,也就是把下面第一行代码改为第二行代码。
将:
LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_hl hdf5
改为:
LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_serial_hl hdf5_serial
错误内容2:
"libcudart.so.8.0 cannot open shared object file: No such file or directory"
解决办法是将一些文件复制到/usr/local/lib文件夹下:
#注意自己CUDA的版本号!
sudo cp /usr/local/cuda-8.0/lib64/libcudart.so.8.0 /usr/local/lib/libcudart.so.8.0 && sudo ldconfig
sudo cp /usr/local/cuda-8.0/lib64/libcublas.so.8.0 /usr/local/lib/libcublas.so.8.0 && sudo ldconfig
sudo cp /usr/local/cuda-8.0/lib64/libcurand.so.8.0 /usr/local/lib/libcurand.so.8.0 && sudo ldconfig
编译caffe问题主要是cudnn的问题,一定要按相应的版本来匹配!!!
8)测试
make runtest -j4
这个4线程容易出错。
显示PASSED就ojbk了。到此caffe配置完毕!
MNIST数据集测试
cd ~/caffe #caffe的根目录
./data/mnist/get_mnist.sh #下载MNIST数据库并解压缩
./examples/mnist/create_mnist.sh #将其转换成Lmdb数据库格式
./examples/mnist/train_lenet.sh #训练网络
这时候不出问题你就可以看到乱七八糟的代码往天上飞,显卡越好飞的越快。最后显示精度和loss。(GTX1050两分钟左右,我的GT 750m 大概18分钟,GTX1080大概就几十秒吧)
训练完成,caffe也就ok了!文章如有错我改。。。