caffe-ssd version ubuntu16.04 GPU installation
Before installing the GPU driver, install the dependency of caffe
1, dependency installation
sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler
sudo apt-get install --no-install-recommends libboost-all-dev
sudo apt-get install libopenblas-dev liblapack-dev libatlas-base-dev
sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev
sudo apt-get install git cmake build-essential
2. After installing the dependency, you need to install the GPU driver.
When installing the tensorflow-gpu version, the driver installation method has been introduced, namely CUDA and cudnn.
Specific method reference[驱动安装方法](https://blog.csdn.net/WSNjiang/article/details/80632777)
3. Download the caffe-SSD version
git clone https://github.com/weiliu89/caffe
4. Switch the ssd mode in the caffe directory
cd caffe
git checkout ssd
5. Change the Makefile.config configuration file
cp Makefile.config.example Makefile.config
gedit Makefile.config
将
#USE_CUDNN := 1
修改成：
USE_CUDNN := 1
将
#OPENCV_VERSION := 3
修改为：
OPENCV_VERSION := 3
将
#WITH_PYTHON_LAYER := 1
修改为
WITH_PYTHON_LAYER := 1
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib
修改为：
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/hdf5/serial
6. Change Makefile
将： LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_hl hdf5
修改为： LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_serial_hl hdf5_serial boost_regex
将： NVCCFLAGS +=-ccbin=$(CXX) -Xcompiler-fPIC $(COMMON_FLAGS)
修改为： NVCCFLAGS += -D_FORCE_INLINES -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)
7. Install Protobuf-2.6.1 and opencv-3.4.4.
`
cd protobuf-2.6.1
./configure --prefix=/usr/local/
make
sudo make install
export PATH=/usr/local/bin/:$PATH
cd opencv-3.4.4
mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/usr/local ..
make
sudo make install #安装
pkg-config --modversion opencv #查看 opencv 版本验证是否安装成功
8. Compile caffe
make all #如果笔记本CPU核较多可以写成 make all -j8 慎用
9. Add environment variables and run test code
export LD_LIBRARY_PATH=/usr/local/lib/:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
sudo make test
sudo make runtest
10. Use python to call caffe
make pycaffe
Due to the different configurations of the servers, some errors will occur when compiling and invoking python. The following is a summary of the errors that occurred in my own compilation
**/usr/bin/ld: /usr/local/lib/libpython2.7.a(abstract.o): relocation R_X86_64_32S against `_Py_NotImplementedStruct' can not be used when making a shared object; recompile with -fPIC
/usr/local/lib/libpython2.7.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
Makefile:566: recipe for target '.build_release/lib/libcaffe.so.1.0.0-rc3' failed
make: *** [.build_release/lib/libcaffe.so.1.0.0-rc3] Error 1**
Solution:
recompile python
cd python
./configure --prefix=/usr/local/ –enable-shared CFLAGS=-fPIC
make
make install
Some systems run –enable-shared CFLAGS=-fPIC and will report an error. I don’t know what the reason is. I ask the god who knows the reason to comment and answer. thank. .
When solving the first error, using -fPIC keeps reporting errors. To solve this problem, Anaconda2 was installed.
You need to change Makefile.config to change the address of calling python to the path in Anaconda. The following is the entire content of Make.config.
## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!
# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1
# CPU-only switch (uncomment to build without GPU support).
#CPU_ONLY := 1
# uncomment to disable IO dependencies and corresponding data layers
# USE_OPENCV := 0
# USE_LEVELDB := 0
# USE_LMDB := 0
# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
# You should not set this flag if you will be reading LMDBs with any
# possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1
# Uncomment if you're using OpenCV 3
OPENCV_VERSION := 3
# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++
# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr
# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the lines after *_35 for compatibility.
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
-gencode arch=compute_20,code=sm_21 \
-gencode arch=compute_30,code=sm_30 \
-gencode arch=compute_35,code=sm_35 \
-gencode arch=compute_50,code=sm_50 \
-gencode arch=compute_52,code=sm_52 \
-gencode arch=compute_61,code=sm_61
# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
# BLAS := atlas
BLAS := open
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
# BLAS_INCLUDE := /path/to/your/blas
# BLAS_LIB := /path/to/your/blas
# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib
# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app
# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
#PYTHON_INCLUDE := /usr/include/python2.7 \
/usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
ANACONDA_HOME := $(HOME)/anaconda2
PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
$(ANACONDA_HOME)/include/python2.7 \
$(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \
# Uncomment to use Python 3 (default is Python 2)
# PYTHON_LIBRARIES := boost_python3 python3.5m
# PYTHON_INCLUDE := /usr/include/python3.5m \
# /usr/lib/python3.5/dist-packages/numpy/core/include
# We need to be able to find libpythonX.X.so or .dylib.
#PYTHON_LIB := /usr/lib
PYTHON_LIB := $(ANACONDA_HOME)/lib
# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib
# Uncomment to support layers written in Python (will link against Python libs)
WITH_PYTHON_LAYER := 1
# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/hdf5/serial
# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib
# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1
# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute
# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1
# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0
# enable pretty build (comment to see full commands)
Q ?= @
LINKFLAGS := -Wl,-rpath,$(HOME)/anaconda2/lib
undefined reference to `inflateValidate@ZLIB_1.2.9'
collect2: error: ld returned 1 exit status
Makefile:619: recipe for target '.build_release/tools/extract_features.bin' failed
make: *** [.build_release/tools/extract_features.bin] Error 1
Solution:
Install Anaconda2.
So there is an undefined reference to `inflateValidate@ZLIB_1.2.9' error.
Add in the last line of Makefile.config
LINKFLAGS := -Wl,-rpath,$(HOME)/anaconda2/lib
3. The following error is reported in make runtest
Warning! ***HDF5 library version mismatched error***
The HDF5 header files used to compile this application do not match
the version used by the HDF5 library to which this application is linked.
Data corruption or segmentation faults may occur if the application continues.
This can happen when an application was compiled by one version of HDF5 but
linked with a different version of static or shared HDF5 library.
You should recompile the application or check your shared library related
settings such as 'LD_LIBRARY_PATH'.
You can, at your own risk, disable this warning by setting the environment
variable 'HDF5_DISABLE_VERSION_CHECK' to a value of '1'.
Setting it to 2 or higher will suppress the warning messages totally.
Headers are 1.10.2, library is 1.8.16
SUMMARY OF THE HDF5 CONFIGURATION
=================================
General Information:
-------------------
HDF5 Version: 1.8.16
Configured on: Tue Aug 28 18:26:31 UTC 2018
Configured by: buildd@lgw01-amd64-024
Configure mode: production
Host system: x86_64-pc-linux-gnu
Uname information: Linux lgw01-amd64-024 4.4.0-128-generic #154-Ubuntu SMP Fri May 25 14:15:18 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Byte sex: little-endian
Libraries: static, shared
Installation point: /usr
Flavor name: serial
Compiling Options:
------------------
Compilation Mode: production
C Compiler: /usr/bin/cc
CFLAGS: -g -O2 -fstack-protector-strong -Wformat -Werror=format-security
H5_CFLAGS: -std=c99 -pedantic -Wall -Wextra -Wundef -Wshadow -Wpointer-arith -Wbad-function-cast -Wcast-qual -Wcast-align -Wwrite-strings -Wconversion -Waggregate-return -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -Wnested-externs -Winline -Wfloat-equal -Wmissing-format-attribute -Wmissing-noreturn -Wpacked -Wdisabled-optimization -Wformat=2 -Wunreachable-code -Wendif-labels -Wdeclaration-after-statement -Wold-style-definition -Winvalid-pch -Wvariadic-macros -Winit-self -Wmissing-include-dirs -Wswitch-default -Wswitch-enum -Wunused-macros -Wunsafe-loop-optimizations -Wc++-compat -Wstrict-overflow -Wlogical-op -Wlarger-than=2048 -Wvla -Wsync-nand -Wframe-larger-than=16384 -Wpacked-bitfield-compat -Wstrict-overflow=5 -Wjump-misses-init -Wunsuffixed-float-constants -Wdouble-promotion -Wsuggest-attribute=const -Wtrampolines -Wstack-usage=8192 -Wvector-operation-performance -Wsuggest-attribute=pure -Wsuggest-attribute=noreturn -Wsuggest-attribute=format -Wdate-time -Wopenmp-simd -Warray-bounds=2 -Wc99-c11-compat -O3 -fstdarg-opt
AM_CFLAGS:
CPPFLAGS: -Wdate-time -D_FORTIFY_SOURCE=2
H5_CPPFLAGS: -D_GNU_SOURCE -D_POSIX_C_SOURCE=200112L -DNDEBUG -UH5_DEBUG_API
AM_CPPFLAGS: -D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE
Shared C Library: yes
Static C Library: yes
Statically Linked Executables: no
LDFLAGS: -Wl,-Bsymbolic-functions -Wl,-z,relro
H5_LDFLAGS: -Wl,--version-script,$(top_srcdir)/debian/map_serial.ver
AM_LDFLAGS:
Extra libraries: -lpthread -lsz -lz -ldl -lm
Archiver: ar
Ranlib: x86_64-linux-gnu-ranlib
Debugged Packages:
API Tracing: no
Languages:
----------
Fortran: yes
Fortran Compiler: /usr/bin/gfortran
Fortran 2003 Compiler: yes
Fortran Flags: -g -O2 -fstack-protector-strong
H5 Fortran Flags:
AM Fortran Flags:
Shared Fortran Library: yes
Static Fortran Library: yes
C++: yes
C++ Compiler: /usr/bin/c++
C++ Flags: -g -O2 -fstack-protector-strong -Wformat -Werror=format-security
H5 C++ Flags:
AM C++ Flags:
Shared C++ Library: yes
Static C++ Library: yes
Features:
---------
Parallel HDF5: no
High Level library: yes
Threadsafety: yes
Default API Mapping: v18
With Deprecated Public Symbols: yes
I/O filters (external): deflate(zlib),szip(encoder)
MPE: no
Direct VFD: no
dmalloc: no
Clear file buffers before write: yes
Using memory checker: no
Function Stack Tracing: no
Strict File Format Checks: no
Optimization Instrumentation: no
Bye...
*** Aborted at 1558417794 (unix time) try "date -d @1558417794" if you are using GNU date ***
PC: @ 0x7f1f8aad3428 gsignal
*** SIGABRT (@0x3e80000506b) received by PID 20587 (TID 0x7f1f733f7980) from PID 20587; stack trace: ***
@ 0x7f1f8ae79390 (unknown)
@ 0x7f1f8aad3428 gsignal
@ 0x7f1f8aad502a abort
@ 0x7f1f90a03290 H5check_version
@ 0x1d88830 (unknown)
Aborted (core dumped)
Makefile:526: recipe for target 'runtest' failed
make: *** [runtest] Error 134
4. When importing caffe:
No module named google.protobuf.internal
Solution:
pip2 install protobuf
NVCC src/caffe/layers/bnll_layer.cu
nvcc fatal : Unsupported gpu architecture 'compute_20'
因为我装的是CUDA9.0所以把下面这两行删除就可以了
[html] view plain copy
-gencode arch=compute_20,code=sm_20 \
-gencode arch=compute_20,code=sm_21 \