Excellent reference: link
Thanks to the author! ! !
Summarize the pits I encountered before writing, a handful of bitter tears! !
Big pit 1:
I checked my /usr/local/include... There is a function of tensor_shape.h in that path. Some people say that tensorflow_cc.so can't find the link . Some people say that link is compiled since tensorflow is compiled. By the way, compile whl, and then uninstall the tensorflow installed by pip install. I tried this method, but still reported a bunch of undefined balabala errors. My final solution is: add -fuse-ld=gold to the linker, which I used last This line of statement is like this: g++ -std=c++11 -fuse-ld=gold…
Tai Hang 2:
./third_party/eigen3/unsupported/Eigen/CXX11/Tensor:1:42: fatal error: unsupported/Eigen/CXX11/Tensor: No such file or directory
The reason for this is most likely that Eigen was not installed or found. I didn’t find it. Da
Hang 3:
This file was generated by an older version of protoc
The reason for this error is that the versions of tensorflow and protocbuf do not match! ! Be sure to install the correct version of protocbuf. I have a feeling that this version of protocbuf and tensorflow correspond well, because I failed to install 3.6.1, only 3.6.0 was installed successfully! !
Pit 4: The
abseil library reported an error. The link solved the problem, but I didn’t report the error again after reconfiguring it. I don’t know if the error disappeared because of the resolution of other errors, or because the directory where the library was located was blocked. The reason why I joined the search database, I didn't report this error again, so please record it first!
Forget about others, the main time is spent on these! ! !
Environment:
ubuntu16.04
python3.5
protocbuf 3.6.0
bazel 0.15.0 NCCL
: 2.2.13
tensorflow-gpu=1.12
Eigen version: 3.3.90
- Install bazel
下载Bazel官网：
https://github.com/bazelbuild/bazel/releases
安装
./bazel-0.15.0-installer-linux-x86_64.sh
- Install protocbuf
The version I installed before was 3.6.1 and then reported the error of version mismatch! Later I found out that the version information was already given in the source code, just install it according to the given version information! The version information is in ./tensorflow/workspace.bzl under the source code tensorflow. The main ones used to compile tensorflow and c++ are Eigen, protocbuf, and bazel. Just remember the version information of these libraries! I found the version I want to use in this file!
官网：
https://github.com/google/protobuf/releases
我下载的版本3.6.0
cd prtobuf-3.6.0
./configure
sudo make -j8
make check -j8
sudo make install
sudo ldconfig
查看版本信息：protoc --version
(这个过程还蛮久的！)
To view the protobuf version installed on this machine:
1. pip show protobuf
2. protoc --version
Here I want to attach a method to uninstall protocbuf:
sudo apt-get remove libprotobuf-dev
which protoc
这样删除我发现并没有删干净
sudo rm /usr/local/bin/protoc //执行文件
sudo rm -rf /usr/local/include/google //头文件
sudo rm -rf /usr/local/lib/libproto* //库文件
这种办法才可以删的干净！！
- Install Eigen, this is the C++ matrix runtime library
官网：
https://gitlab.com/libeigen/eigen/-/releases
我的版本3.3.90
tar -zxvf eigen-3.3.90.tar.gz
cd eigen-3.3.90/
mkdir build
cd build
cmake ..
sudo make
sudo make install
sudo cp -r /usr/local/include/eigen3/Eigen /usr/local/include
After the installation is complete, you can search for a small program online to test whether the installation is successful. Thank you for helping me answer why I need to copy it to the include library at the end, with a link link
sudo cp -r /usr/local/include/eigen3/Eigen /usr/local/include
4. Install NCCL
官网：
https://developer.nvidia.com/nccl
我下载的2.2.13(给我眼睛都找瞎了才找到)
sudo dpkg -i nccl-repo-ubuntu1604-2.2.13-ga-cuda9.0_1-1_amd64.deb
sudo apt update
sudo apt install libnccl2 libnccl-dev
sudo cp /usr/lib/x86_64-linux-gnu/libnccl.so.2 /usr/local/cuda/lib
sudo cp /usr/include/nccl.h /usr/local/cuda/include/
- To compile tensorflow,
I have the source code version 1.12
官网:
https://github.com/tensorflow/tensorflow/tags
tips:
这里也可以用git方法，然后切换分支，但是我更喜欢源码安装，就没选那种方法
./configure
这个过程我只记得一些没选默认的：
1. 我的ubuntu上有python3.5，python3.6，我想安装在3.5下，我就指定了安装目录:/usr/bin/python3.5
2. cuda那个我选了Y，因为我要用cuda，其他的都是默认或者n
The following is compiled
正常人都是这么编译的：
bazel build --config=opt //tensorflow:libtensorflow_cc.so // 无显卡，cpu版本
bazel build --config=opt --config=cuda //tensorflow:libtensorflow_cc.so // 有显卡
我看到人家说要加个参数，不然c++ imread()读图容易出问题，但是我搜了很久都没搜到人家混合一起的命令，可我还是加了，我的命令是这样的
bazel build --config=monolithic --config=cuda //tensorflow:libtensorflow_cc.so
这个过程差不多20多分钟吧，看到successful就是成功了，我编译了好几次，这边都没出过错！
后面我要pip install .whl所以这边我又编译了whl
bazel build --config=noaws //tensorflow/tools/pip_package:build_pip_package --verbose_failures --copt=-funsafe-math-optimizations --copt=-march=armv8-a --copt=-mtune=cortex-a53
When I get here, my blog will say to execute this script./tensorflow/contrib/makefile/download_dependencies.sh to download the things I need, I didn't succeed! Fortunately, I later found out that these libraries can all be downloaded by myself!
But here I performed another operation, because I always report an error
/usr/local/include/tf/tensorflow/core/public/session.h:22:60: fatal error: tensorflow/core/framework/device_attributes.pb.h: No such file or directory
I took a look at the file device_attributes.pb.h under bazel-genfiles/tensorflow/core/framework! Later, some errors were reported. I found it under bazel-genfiles/tensorflow/cc/ops. Then I overwritten the contents of the cc and core folders in "tensorflow/bazel-genfiles/tensorflow/". tensorflow/ tensorflow/file, hands are shaking when knocking! ! ! But then no error was reported, hahahahaha! ! !
Then build folders and move files
sudo mkdir /usr/local/include/tf
sudo cp -r bazel-genfiles/ /usr/local/include/tf/
sudo cp -r tensorflow /usr/local/include/tf/
sudo cp -r third_party /usr/local/include/tf/
sudo cp bazel-bin/tensorflow/libtensorflow_cc.so /usr/local/lib/
sudo cp bazel-bin/tensorflow/libtensorflow_framework.so /usr/local/lib
My test environment is using this project link
Here I add the library path to the search directory, that is, add the library directory to the back of the file /etc/ld.so.conf.
Then ldconfig
ldconfig -v shows the path just added. If an error is reported, ibtensorflow_cc.so: No such file or directory , That is, the file moved above is not right, check the part of the file moved above, until gcc -ltensorflow_cc --verbose doesn't report the error that tensorflow_cc.so can't be found, it succeeds!
I am a little bit different from the link I referred to. My script here is like this:
SOURCE_DIR =./src
BIN_DIR = ./bin
CPP = g++ -std=c++11 -fuse-ld=gold
LDFLAGS = -g -Wall -D_DEBUG -Wshadow -Wno-sign-compare -w
LDLIBS = -L/usr/local/lib/
INCLUDES = -I/usr/local/include/tf -I/usr/local/include/eigen3
ACTUAL_LIBS = `pkg-config --cflags --libs protobuf` -ltensorflow_cc -ltensorflow_framework
INPUT_FILE = $(SOURCE_DIR)/main.cpp $(SOURCE_DIR)/ann_model_loader.cpp
OBJET_FILE = $(BIN_DIR)/tfcpp_demo
tfcpp_demo:
$(CPP) -o $(OBJET_FILE) $(INCLUDES) $(LDFLAGS) $(LDLIBS) $(ACTUAL_LIBS) $(INPUT_FILE)
After make, run sh run.sh and output the result. So far, you are done! ! !
Here my ubuntu default gcc version is 4.8, but I saw the official website said that the gcc version required for ubuntu16.04 is 4.8, I tossed again, switch the gcc version, refer to the link link and notice that others are combined together, one command Solved, but I can only separate
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 60 --slave /usr/bin/g++ g++ /usr/bin/g++-4.8
下面是我的：
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 101
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.8 101
g++ is the same operation! !
I need to attach a small episode here, because I keep reporting the error of make, which is the undefined error above. I checked my other things and there is no problem. The difference from others is the command when compiling tensorflow_cc.so, but there is no error. I I don't want to recompile! Then I saw that the cmake succeeded in ubuntu. I thought I didn’t get it right now, so I might as well try it, and I saw this blog link . This person is also very happy. Running this script is successful./download_dependencies .sh, envious face! ! His previous section is the same with me, I have no control, then that part of me is starting, is
followed by the establishment of a CMakeList.txt file in the folder, write this part of the beginning of the file,
create a new file CMakeLists.txt
I I copied the source code of my tensorflow1.12 to the main.cpp file, and now there are these files in this path:
cmake_minimum_required(VERSION 3.11)
project(cpptensorflow)
set(CMAKE_CXX_STANDARD 11)
link_directories(/home/ubuntu/tensorflow-1.12.0/bazel-bin/tensorflow)
include_directories(
/home/ubuntu/tensorflow-tutorial/cpp/src/tensorflow-1.12.0
/home/ubuntu/tensorflow-tutorial/cpp/src/tensorflow-1.12.0/bazel-genfiles
/home/ubuntu/tensorflow-tutorial/cpp/src/tensorflow-1.12.0/bazel-bin/tensorflow
/usr/local/include/eigen3
)
add_executable(cpptensorflow main.cpp ann_model_loader.h model_loader_base.h ann_model_loader.cpp)
target_link_libraries(cpptensorflow tensorflow_cc tensorflow_framework)
Something happened here. My cmake version is 3.6. I told me that the version is low and I need to re-download the high version. I saw this link. This person must be a great god. There is no nonsense at all. The words are concise and the installation is successful! ! This great god is really bullish! ! ! Backhand is just a concern, I look forward to more updates, let me learn more! ! ! !
Then compile the project
mkdir build
cd build
cmake ..
make
Then run
./cpptensorflow /home/ubuntu/tensorflow-tutorial/cpp/model/nn_model_frozen.pb
At this point, it was successful. It also showed that my tensorflow_cc.so file was compiled without problems, and then I started looking for the reason of undefined. Later, I found that g++ compiled the linker with -fuse-ld=gold and it was just fine, and the desired output was successfully output. result! !
This time is really successful!
Of course, my own project has also been successfully configured, and I wrote it to record my own bugs and some experiences with other people's projects! !