Building a Machine Learning Software Platform

Please use a VPN when doing the following

Install the necessary support software

sudo apt-get update
sudo apt-get upgrade

Install toolkits such as images, videos and HMI

sudo apt-get install build-essential cmake git unzip pkg-config
sudo apt-get install libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
sudo apt-get install libxvidcore-dev libx264-dev
sudo apt-get install libgtk-3-dev
sudo apt-get install libhdf5-serial-dev graphviz
sudo apt-get install libopenblas-dev libatlas-base-dev gfortran
sudo apt-get install python-tk python3-tk python-imaging-tk
sudo apt-get install build-essential
sudo apt-get install cmake git unzip zip
sudo apt-get install python2.7-dev python3.5-dev python3.6-dev pylint


Start to create a virtualenv independent environment to deal with the problem that different projects require different versions of software

wget https://bootstrap.pypa.io/get-pip.py
sudo python get-pip.py
sudo python3 get-pip.py
sudo pip install virtualenv virtualenvwrapper
sudo rm -rf ~/.cache/pip get-pip.py

After installing virtualenv and virtualenvwrapper , update the ~ / .bashrc file and add the following lines at the end of the document

# virtualenv and virtualenvwrapper
export WORKON_HOME=$HOME/.virtualenvs
export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3
source /usr/local/bin/virtualenvwrapper.sh

Create 'ruzw' virtual working environment and install numpy in 'ruzw'

source ~/.bashrc
mkvirtualenv ruzw -p python3
workon ruzw
pip install numpy

Compile and install OpenCV

cd ~
wget -O opencv.zip https://github.com/Itseez/opencv/archive/3.3.0.zip
wget -O opencv_contrib.zip https://github.com/Itseez/opencv_contrib/archive/3.3.0.zip
unzip opencv.zip
unzip opencv_contrib.zip

Create a build path for CMake operations

cd ~/opencv-3.3.0/
mkdir build
cd build

Enter the following commands:

cmake -D CMAKE_BUILD_TYPE=RELEASE \
    -D CMAKE_INSTALL_PREFIX=/usr/local \
    -D WITH_CUDA=OFF \
    -D INSTALL_PYTHON_EXAMPLES=ON \
    -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib-3.3.0/modules \
    -D BUILD_EXAMPLES=ON ..

Now compile OpenCV

make -j4

The next step is to install OpenCV3.3

sudo make install
sudo ldconfig
cd ~

Link OpenCV and virtual environment 'ruzw'

cd ~/.virtualenvs/ruzw/lib/python3.5/site-packages/
ln -s /usr/local/lib/python3.5/dist-packages/cv2.cpython-35m-x86_64-linux-gnu.so cv2.so
cd ~

Test OpenCV installation and linking

workon ruzw
python
import cv2
cv2.__version__
'3.3.0'

安装TensorFlow(tensorflow 1.7.0 GPU、CUDA Toolkit 9.1 、cuDNN 7.1.2

First test if there is an NVIDIA GPU locally

lspci | grep -i nvidia

Test the linux version (x86_64 indicates that the system is a 64-bit system, supported by cuda 9.1)

uname -m && cat /etc/*release

Heads up to install the linux kernel

uname -r
sudo apt-get install linux-headers-$(uname -r)

Download and install NVIDIA KUDA Toolkit

wget http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_9.1.85-1_amd64.deb
sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub
sudo dpkg -i cuda-repo-ubuntu1604_9.1.85-1_amd64.deb
sudo apt-get update
sudo apt-get install cuda-9.1

Reboot the system to load the Nvidia driver

reboot

Edit ~/.bashrc file

vim ~/.bashrc

add on the last line

export PATH=/usr/local/cuda-9.1/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-9.1/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

Test drive version

source ~/.bashrc
sudo ldconfig
nvidia-smi

Log in to the registration website https://developer.nvidia.com/cudnn to download the following files

cuDNN v7.1.2 Runtime Library for Ubuntu16.04 (Deb)

cuDNN v7.1.2 Developer Library for Ubuntu16.04 (Deb)

cuDNN v7.1.2 Code Samples and User Guide for Ubuntu16.04 (Deb)

Go to the download file folder in the terminal and do the following

sudo dpkg -i libcudnn7_7.1.2.21-1+cuda9.1_amd64.deb
sudo dpkg -i libcudnn7-dev_7.1.2.21-1+cuda9.1_amd64.deb
sudo dpkg -i libcudnn7-doc_7.1.2.21-1+cuda9.1_amd64.deb

Install confirmation cuDNN

cp -r /usr/src/cudnn_samples_v7/ $HOME
cd  $HOME/cudnn_samples_v7/mnistCUDNN
make clean && make
./mnistCUDNN

Install libcupti (required)

sudo apt-get install libcupti-dev
echo 'export LD_LIBRARY_PATH=/usr/local/cuda/extras/CUPTI/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc

Install Bazel (required)

sudo apt-get install openjdk-8-jdk
wget https://github.com/bazelbuild/bazel/releases/download/0.11.1/bazel_0.11.1-linux-x86_64.deb
sudo dpkg -i bazel_0.11.1-linux-x86_64.deb

Python3.0 version

sudo apt-get install python3-numpy python3-dev python3-pip python3-wheel

Install TensorFlow

source ~/.bashrc
sudo ldconfig
wget https://github.com/tensorflow/tensorflow/archive/v1.7.0.zip
unzip v1.7.0.zip
cd tensorflow-1.7.0
./configure

Set Python default address

Please specify the location of python. [Default is /usr/bin/python]: /usr/bin/python3

Complete the following environment settings

Do you wish to build TensorFlow with jemalloc as malloc support? [Y/n]: Y
Do you wish to build TensorFlow with Google Cloud Platform support? [Y/n]: Y
Do you wish to build TensorFlow with Hadoop File System support? [Y/n]: Y
Do you wish to build TensorFlow with Amazon S3 File System support? [Y/n]: Y
Do you wish to build TensorFlow with Apache Kafka Platform support? [y/N]: N
Do you wish to build TensorFlow with XLA JIT support? [y/N]: N
Do you wish to build TensorFlow with GDR support? [y/N]: N
Do you wish to build TensorFlow with VERBS support? [y/N]: N
Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: N
Do you wish to build TensorFlow with CUDA support? [y/N]: Y
Please specify the CUDA SDK version you want to use, e.g. 7.0. [Leave empty to default to CUDA 9.0]: 9.1
Please specify the location where CUDA 9.1 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: /usr/local/cuda
Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 7.0]: 7.1.2
Please specify the location where cuDNN 7 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: /usr/lib/x86_64-linux-gnu
Do you wish to build TensorFlow with TensorRT support? [y/N]: N
Please note that each additional compute capability significantly increases your build time and binary size. [Default is: 5.0] 5.0
Do you want to use clang as CUDA compiler? [y/N]: N
Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]: /usr/bin/gcc
Do you wish to build TensorFlow with MPI support? [y/N]: N
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]: -march=native
Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]:N

Install TensorFlow with Bazel (under the tensorflow-1.7.0 path, it may last 1~2 hours)

sudo ln -s /usr/local/cuda/include/crt/math_functions.hpp /usr/local/cuda/include/math_functions.hpp
bazel build --config=opt --config=cuda --incompatible_load_argument_is_label=false //tensorflow/tools/pip_package:build_pip_package
bazel-bin/tensorflow/tools/pip_package/build_pip_package tensorflow_pkg

Activate virtual environment (python3)

cd tensorflow_pkg
pip3 install tensorflow*.whl

Test TensorFlow installation results

python

import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sex = tf.Session ()
print (sess.run (hello))

If the following results appear, TensorFlow is installed successfully

b'Hello, TensorFlow'

Install Keras

First make sure you are in a virtual environment (ruzw)

Install the necessary basic software

pip install scipy matplotlib pillow
pip install imutils h5py requests progressbar2
pip install scikit-learn scikit-image

Install Keras

pip install hard
Get familiar with the ~ / .keras / keras.json file, make sure image_data_format is set to channels_last and backend is set to tensorflow
{
    "image_data_format": "channels_last",
    "backend": "tensorflow",
    "epsilon": 1e-07,
    "floatx": "float32"
}

install mxnet

Copy the 0.11.0 branch in mxnet

cd ~
git clone --recursive https://github.com/apache/incubator-mxnet.git mxnet --branch 0.11.0
cd mxnet
make -j4 USE_OPENCV=1 USE_BLAS=openblas USE_CUDA=1 USE_CUDA_PATH=/usr/local/cuda USE_CUDNN=1

Link to our ruzw virtual environment

cd ~/.virtualenvs/ruzw/lib/python3.5/site-packages/
ln -s ~/mxnet/python/mxnet mxnet
cd ~

test mxnet

python
import mxnet
Note that mxnet documents cannot be deleted!



Guess you like

Origin http://43.154.161.224:23101/article/api/json?id=324692700&siteId=291194637