background
Forced to start the topic, I wanted to study the content of the dynamic knowledge graph. I was just in time to read an article dyngraph2vec: Capturing Network Dynamics using Dynamic Graph Representation Learning , which open sourced the library DynGEM based on my own article . In line with the principle of utilitarianism, I started the process of exploring it last week
According to the README.md of this library, you must first install DynTriad and its dependent environment, and then install DynGEM itself and its environment. I thought that the installation of the library should not be difficult. I expected that because many dependencies require source installation, and even if the installation is completed, it can be imported, but due to the dependency version, environment variables and library code itself, some source code needs to be modified. In order to run smoothly, it took some setbacks, and it took 5 days to finish. Therefore, here is the record of the installation and debugging process under CentOS7, I hope you will avoid detours.
DynTriad
The following steps are recommended to be carried out in order
Boost.Python
To download and install the 1.58 version and 1.73 version , then the commands are the same, just need to change 58 to 73.
Take 1.73 as an example, run the command to decompress and cut in
[root@scentos DynamicGraph]# tar -xvzf boost_1_73_0.tar.gz
[root@scentos DynamicGraph]# cd boost_1_73_0/Then run the command to compile and install
[root@scentos boost_1_73_0]# ./bootstrap.sh
[root@scentos boost_1_73_0]# ./b2 installWait for it to complete. After completion, a lot of libraries will be generated under /usr/local/lib
[root@scentos boost_1_73_0]# ll/usr/local/lib
total 42032
.....If you need to compile and install the python3 version, follow the steps below:
First modify the bootstrap.sh file and modify the content under python.configured. Comment out the original using python: $PYTHON_VERSION: "$PYTHON_ROOT"; and write the content of Python3 according to the cat. Using python: 3.5: "/usr/local/bin/python3.5": /usr/local/include/python3.5m : /usr/local/lib;
import python ;
if ! [ python.configured ]
{
# using python : $PYTHON_VERSION : "$PYTHON_ROOT" ;
# using python : 3.6 : "/usr/bin/python3.6" : /usr/include/python3.6m : /usr/local/lib ;
using python : 3.5 : "/usr/local/bin/python3.5" : /usr/local/include/python3.5m : /usr/local/lib ;
}
EOF
fiThen call the bootstrap script and add parameters
[root@scentos boost_1_73_0]# ./bootstrap.sh --with-python=python3 --with-python-root=/usr/local/bin/python3.5 --with-python-version=3.5The installation command is still ./b2 install.
After trial and error, I found that version 1.73 was compiled with Python 3.6 and Python 2.7, and version 1.58 was compiled with Python 3.5
CMake
Download the package https://cmake.org/download/ , and then run the command
[root@scentos DynamicGraph]# tar -zxvf cmake-3.18.2.tar.gz
[root@scentos DynamicGraph]# cd cmake-3.18.2/
[root@scentos cmake-3.18.2]# ./bootstrap
[root@scentos cmake-3.18.2]# make
[root@scentos cmake-3.18.2]# make installown
Download the package http://eigen.tuxfamily.org/index.php?title=Main_Page#Download , then unzip
[root@scentos DynamicGraph]# tar -zxvf eigen-3.3.7.tar.gzDon't compile and install in the unzipped eigen-3.3.7 directory, you have to create a new directory (here named eigen3.3.7), then go into this directory, and compile and install eigen here
[root@scentos DynamicGraph]# mkdir eigen3.3.7
[root@scentos DynamicGraph]# cd eigen3.3.7
[root@scentos eigen3.3.7]# cmake ../eigen-3.3.7
[root@scentos eigen3.3.7]# make installmpfr and expat
The download path is https://www.mpfr.org/mpfr-current/#download and https://sourceforge.net/projects/expat/ , after downloading, decompress, enter the two directories, and execute them respectively
./configure
make && make installcgal
Download path https://github.com/CGAL/cgal/tree/releases/CGAL-4.7 , download and unzip, enter the directory, and execute the command
[root@scentos cgal-releases-CGAL-4.7]# cmake .
[root@scentos cgal-releases-CGAL-4.7]# make && make installGraphTool
Download the project compressed package https://git.skewed.de/count0/graph-tool/-/tree/master , first make sure that your gcc version>=9.1.0
[root@scentos DynamicGraph]# gcc --version
gcc (GCC) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.If the version is relatively low, or there is no gcc at all, then you can refer to my article about upgrading gcc to 9.3.0 under CentOS7 to upgrade or install
Then install the pip of python2, download the path https://pypi.org/project/pip/#files , enter the corresponding directory after decompression, and execute the command
[root@scentos pip-20.2.3]# python setup.py install
[root@scentos pip-20.2.3]# pip install pycairo scipyUse yum to install some dependencies
yum -y install numpy.x86_64 scipy.x86_64Finally, run the command to compile and install graph-tool. It will take a long time here. It took more than two hours for my virtual machine to have a quad-core 8G memory.
[root@scentos DynamicGraph]# tar -zxvf graph-tool-master.tar.gz
[root@scentos DynamicGraph]# cd graph-tool-master/
[root@scentos graph-tool-master]# ./autogen.sh
[root@scentos graph-tool-master]# ./configure --disable-sparsehash
[root@scentos graph-tool-master]# make && make installDynamicTriad
After installing the above library, you can compile and install DynamicTriad, the project address is https://github.com/luckiezhou/DynamicTriad
First install dill with pip
[root@scentos DynamicTriad-master]# pip install dillThen perform the installation, the following path and other variables should be changed to your own
[root@scentos DynamicTriad-master]# ./build.sh
entering /home/szc/DynamicGraph/DynamicTriad-master
You may need to specify some environments before building
PYTHON_LIBRARY? (default: /usr/lib64/libpython2.7.so.1.0, use a space ' ' to leave it empty)
PYTHON_INCLUDE_DIR? (default: /usr/include/python2.7, use a space ' ' to leave it empty)
EIGEN3_INCLUDE_DIR? (default: /usr/include, use a space ' ' to leave it empty) /home/szc/DynamicGraph/eigen-3.3.7
BOOST_ROOT? (default: , use a space ' ' to leave it empty) /usr/local/lib
name for boost_python library? (useful when boost_python cannot be detected by cmake) (default: boost_python, use a space ' ' to leave it empty) Some variable paths inside (/home/szc/DynamicGraph/eigen-3.3.7, /usr/local/lib) change according to your own situation
The results are as follows
Then run test.py to see if the build is complete
At this point, DynTriad and its dependencies are installed and stopped
DynGEM
In addition to DynTriad, this library also has to install MATLAB and LaTex
MATLAB
Download address https://www.mathworks.com/products/compiler/matlab-runtime.html , select the download of 2019 (9.6). After downloading and decompressing, run ./install, and then default all the way. Modify the /etc/profile file after completion, and add a line at the end
export LD_LIBRARY_PATH=/usr/local/lib:/usr/local/MATLAB/MATLAB_Runtime/v96/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v96/runtime/glnxa64:/usr/local/MATLAB/MATLAV_Runtime/v96/sys/os/glnxa64After saving and exiting, source it to make it effective
LaTex
The download address is https://mirrors.tuna.tsinghua.edu.cn/CTAN/systems/texlive/Images/ and select textlive.iso inside. After downloading, put it in the virtual machine, clear the original one, mount the new one, and install it
[root@scentos szc]# locate texlive | xargs rm -rf
[root@scentos szc]# mount -o loop texlive.iso /mnt/
[root@scentos szc]# cd /mnt/
[root@scentos mnt]# ./install-tlChoose the installation method and choose i directly. After installation, add a few lines of environment variables to the end of the /etc/profile file
export PATH=/usr/local/texlive/2020/bin/x86_64-linux:$PATH
export MANPATH=/usr/local/texlive/2020/texmf-dist/doc/man
export INFOPATH=/usr/local/texlive/2020/texmf-dist/doc/infoThen enable the source, check the latex version
[root@scentos mnt]# source /etc/profile
[root@scentos mnt]# tex --version
TeX 3.14159265 (TeX Live 2020)
kpathsea version 6.3.2
Copyright 2020 D.E. Knuth.
There is NO warranty. Redistribution of this software is
covered by the terms of both the TeX copyright and
the Lesser GNU General Public License.
For more information about these matters, see the file
named COPYING and the TeX source.
Primary author of TeX: D.E. Knuth.
[root@scentos mnt]#DynGEM
The project address is https://github.com/palash1992/DynamicGEM , after downloading and decompressing into the directory, run directly
python setup.py installYes, if an error is reported
Traceback (most recent call last):
File "./setup.py", line 185, in <module>
setup_package()
File "./setup.py", line 138, in setup_package
write_version_py()
File "./setup.py", line 119, in write_version_py
version = imp.load_source('DynamicGEM.version', 'dynamicgem/version.py')
File "/usr/lib64/python3.6/imp.py", line 172, in load_source
module = _load(spec)
File "<frozen importlib._bootstrap>", line 684, in _load
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 674, in exec_module
File "<frozen importlib._bootstrap_external>", line 781, in get_code
File "<frozen importlib._bootstrap_external>", line 741, in source_to_code
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "dynamicgem/version.py", line 3
short_version = '1.0.0'Then modify the dynamicgem/version.py file, clear the original content, and copy the following content (this is actually caused by the indentation of each line of code in the version.py generated by the code).
short_version = '1.0.0'
version = '1.0.0'
full_version = '1.0.0'
git_revision = 'Unknown'
release = True
if not release:
version = full_version
short_version += ".dev"Debug DynGEM
Using pycharm, paste the test code in the README.md of DynGEM official website with python3.6 and run it, and found the following problems:
1). Since the mygraph in the pre-compiled DynamicTraid is the so package, the method of importing it in DynamicGEM is wrong, and it needs to be changed to the importing method of ctype:
# /usr/local/lib/python3.6/site-packages/dynamicgem/dynamictriad/core/gconv.py
import ctypes
import os
# mygraph=ctypes.cdll.LoadLibrary(os.path.realpath('')+'/dynamicgem/dynamictriad/core/mygraph.so')
dll_path_ = "/home/szc/DynamicGraph/DynamicGEM-master/dynamicgem/dynamictriad/core/mygraph-build/mygraph.so"
mygraph = ctypes.cdll.LoadLibrary(dll_path_)
# import dynamicgem.dynamictriad.core.mygraph as mygraphas well as
# /usr/local/lib/python3.6/site-packages/dynamicgem/dynamictriad/core/mygraph_utils.py
import os
import ctypes
# mygraph=ctypes.cdll.LoadLibrary(os.path.realpath('')+'/dynamicgem/dynamictriad/core/mygraph.so')
dll_path_ = "/home/szc/DynamicGraph/DynamicGEM-master/dynamicgem/dynamictriad/core/mygraph-build/mygraph.so"
mygraph = ctypes.cdll.LoadLibrary(dll_path_)
# import dynamicgem.dynamictriad.core.mygraph as mygraph2) Replace the libstdc++ file of MATLAB. This is because there is no GLIBCXX_3.4.26 in the 6.0.22 version of the libstdc++.so file of MATLAB9.6, but there is in the 6.0.28 version, so you need to replace it with the latter, otherwise just wait for an error.
[root@scentos boost_1_73_0]# mv /usr/local/MATLAB/MATLAB_Runtime/v96/sys/os/glnxa64/libstdc++.so.6.0.22 /usr/local/MATLAB/MATLAB_Runtime/v96/sys/os/glnxa64/libstdc++.so.6.0.22_old
[root@scentos boost_1_73_0]# cp /usr/lib64/libstdc++.so.6.0.28 /usr/local/MATLAB/MATLAB_Runtime/v96/sys/os/glnxa64/
[root@scentos boost_1_73_0]# rm -f /usr/local/MATLAB/MATLAB_Runtime/v96/sys/os/glnxa64/libstdc++.so.6
[root@scentos boost_1_73_0]# ln -s libstdc++.so.6.0.28 /usr/local/MATLAB/MATLAB_Runtime/v96/sys/os/glnxa64/libstdc++.so.63). Ensure that the versions of keras and tensorflow match, where tensorflow should be version 1.11, and modify the keras version according to https://docs.floydhub.com/guides/environments/
[root@scentos Textset]# /usr/local/bin/pip3.6 install tensorflow==1.11
[root@scentos Textset]# /usr/local/bin/pip3.6 install keras==2.2.4Then upgrade numpy, modify the h5py version
[root@scentos Textset]# /usr/local/bin/pip3.6 install -U numpy
[root@scentos Textset]# /usr/local/bin/pip3.6 install h5py==2.8.04). If you still encounter the problem of super-parameter non-equivalence during operation, you must skip to the corresponding code part to modify it.
For example, I am running to the address of MATLAB in the environment variable, so I modified the corresponding part (if not os.environ) in the corresponding file (/usr/local/lib/python3.6/site-packages/TIMERS_ALL/__init__.py) [self.path_var].__contains__("MATLAB") part):
def get_paths_from_os(self):
....
path_elements = []
if self.path_var in os.environ:
if not os.environ[self.path_var].__contains__("MATLAB"):
os.environ[self.path_var] = os.environ[self.path_var] + os.pathsep\
+ "/usr/local/MATLAB/MATLAB_Runtime/v96/bin/glnxa64:" \
"/usr/local/MATLAB/MATLAB_Runtime/v96/runtime/glnxa64:" \
"/usr/local/MATLAB/MATLAB_Runtime/v96/sys/os/glnxa64"
path_elements = os.environ[self.path_var].split(os.pathsep)
if not path_elements:
.....It can also be specified in Edit Configuration->Environment Vairables
5). Going here, in fact, most of DynGEM's code can run, but DynTriad still has problems
....
File "/usr/local/lib/python3.6/site-packages/dynamicgem/dynamictriad/core/mygraph_utils.py", line 90, in load_adjlist
g = mygraph.Graph(node_type, weight_type)
File "/usr/lib64/python3.6/ctypes/__init__.py", line 356, in __getattr__
func = self.__getitem__(name)
File "/usr/lib64/python3.6/ctypes/__init__.py", line 361, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /...../DynamicGEM-master/dynamicgem/dynamictriad/core/mygraph-build/mygraph.so: undefined symbol: GraphThe corresponding code is
# dynamicgem\dynamictriad\core\mygraph_utils.py
def load_adjlist(fn, node_type='string', weight_type='float'):
"""
loads only undirected graph, if multiple instances of the same edge is detected,
their weights are summed up
:param fn:
:param node_type:
:param weight_type:
:return:
"""
py_node_type = type2python(node_type)
py_weight_type = type2python(weight_type)
edgeset = set() # check if the graph is undirected
g = mygraph.Graph(node_type, weight_type)
for line in open(fn, 'r'):
fields = line.split()According to the error message, the error is that there is no Graph attribute or class in mygraph.so, and I changed the so package path from DynGEM to DynTriad to compile it. There is still this problem, but the dyngem in site-packages There is really no mygraph.py file in the /dyntraid/core directory, so this should be a problem with the DynGEM code itself. I have already raised an issue to the author, but I haven’t seen a reply yet.
operation result
Taking the signature model AERNN in the article as an example, the test runs DynGEM, and the run ends after more than 40 minutes. The screenshot is as follows:
Conclusion
At this point, even if DynGEM is installed and can run normally (except for the DynTriad test part), we can understand and explore it by means of breakpoint debugging.
Finally, attach all the links to the network disks for downloading things:
Link: https://pan.baidu.com/s/1TaDTq555xIF6o_xWGlHq1Q
Extraction code: kkwf