Tinkerpop + JanusGraph + Hbase simple implementation
〇, machine environment description
192.168.1.2 master Install and configure jdk1.8; have a normal running Hadoop+Hbase+Zookeeper cluster. ... Customize other cluster information.
Note: The host IP, host name and installation directory in the document are for reference only, please adjust it according to the actual environment.
Apache TinkerPop™ is a graph computing framework for graph databases (OLTP) and graph analytics systems (OLAP).
JanusGraph is a scalable graph database designed for storing and querying graphs containing hundreds of billions of vertices and edges distributed across multi-machine clusters. JanusGraph is a transactional database that can support thousands of concurrent users to perform complex graph traversals in real time.
1. Cluster construction of Hadoop+Hbase+Zookeeper
Although this process is not troublesome, it is still time-consuming to explain it clearly, so I will omit it here.
2. Create a directory and download Tinkerpop
According to the information provided by the official JanusGraph document <<Table B.1. Version Compatibility Matrix>>, in this construction, Tinkerpop 3.2.7, JanusGraph 0.2.0 are used, and the relevant address is http://docs.janusgraph.org/latest /version-compat.html.
1. Create a directory
[root@master ~]# mkdir -p /usr/local/program [root@master ~]# cd /usr/local/program/ [root@master program]# pwd /usr/local/programe2. Download Tinkerpop
[root@master program]# wget http://www-eu.apache.org/dist/tinkerpop/3.2.7/apache-tinkerpop-gremlin-console-3.2.7-bin.zip # console, related address http: //tinkerpop.apache.org/. [root@master program]# wget http://mirror.bit.edu.cn/apache/tinkerpop/3.2.7/apache-tinkerpop-gremlin-server-3.2.7-bin.zip # server, related address http: //tinkerpop.apache.org/.3. Unzip Tinkerpop
[root@master program]# unzip apache-tinkerpop-gremlin-server-3.2.7-bin.zip [root@master program]# unzip apache-tinkerpop-gremlin-console-3.2.7-bin.zip [root@master program]# ls -1 apache-tinkerpop-gremlin-console-3.2.7 apache-tinkerpop-gremlin-server-3.2.7 apache-tinkerpop-gremlin-console-3.2.7-bin.zip apache-tinkerpop-gremlin-server-3.2.7-bin.zip
3. Install JanusGraph dependencies on Tinkerpop Server
Note: Section 4 describes how to configure grapeConfig.xml. If the download fails, please refer to Section 4.
1. Enter
apache-tinkerpop-gremlin-server-3.2.7 to
download dependencies
Installation method: bin/gremlin-server.sh -i org.janusgraph janusgraph-all $VERSION, download JanusGraph version 0.2.0 here.
[root@master apache-tinkerpop-gremlin-server-3.2.7]# bin/gremlin-server.sh -i org.janusgraph janusgraph-all 0.2.02. Important note (Important, this place should be highlighted, in order to look good, that's all)
Here's a quote from 7.4.2. Using TinkerPop Gremlin Server with JanusGraph in the official JanusGraph document:
The above command uses Groovy Grape and if it is not configured properly download errors may ensue. Please refer to this section of the TinkerPop documentation for more information around setting up ~/.groovy/grapeConfig.xml.
Relevant address
http://docs.janusgraph.org/latest/server.html.
The gist is:
The above command uses Groovy Grape, if not properly configured, download errors may occur. For more information on the settings, see this section of the TinkerPop documentation ~/.groovy/grapeConfig.xml.
The address mentioned above
http://tinkerpop.apache.org/docs/3.2.6/reference/#gremlin-applications.
Fourth, in order to avoid the failure of downloading dependencies, modify the Groovy Grape
Note: Windows directory: C:\Users\[User_Name]\.groovy; Linux directory: ~/.groovy
The modifications are as follows:
1. Create the grapeConfig.xml file in the above directory, or if the directory does not exist, create the directory first.
2. Modify the contents of the file as follows:
<ivysettings>
<settings defaultResolver="downloadGrapes"/>
<property name="m2-pattern" value="${user.home}/.m2/repository/org/apache/tinkerpop/[module]/[revision]/[module]-[revision](-[classifier]).[ext]" />
<property name="m2-pattern-ivy" value="${user.home}/.m2/repository/org/apache/tinkerpop/[module]/[revision]/[module]-[revision](-[classifier]).pom" />
<caches>
<cache name="nocache" useOrigin="true" />
</caches>
<resolvers>
<chain name="downloadGrapes">
<filesystem name="local-maven2" checkmodified="true" changingPattern=".*" changingMatcher="regexp" m2compatible="true" cache="nocache">
<artifact pattern="${m2-pattern}"/>
<ivy pattern="${m2-pattern-ivy}"/>
</filesystem>
<filesystem name="cachedGrapes">
<ivy pattern="${user.home}/.groovy/grapes/[organisation]/[module]/ivy-[revision].xml"/>
<artifact pattern="${user.home}/.groovy/grapes/[organisation]/[module]/[type]s/[artifact]-[revision].[ext]"/>
</filesystem>
<ibiblio name="ibiblio" m2compatible="true"/>
<ibiblio name="local" root="file:${user.home}/.m2/repository/" m2compatible="true"/>
<ibiblio name="oracle" root="http://download.oracle.com/maven" m2compatible="true"/>
</chain>
</resolvers>
</ivysettings>
3. 重新执行第三节的内容.
五、参数设置(master节点操作)
1. 在${TINKERPOP_HOME}/conf/目录创建janusgraph-hbase-server.properties, 添加内容如下:
gremlin.graph=org.janusgraph.core.JanusGraphFactory storage.backend=hbase # 存储方式为Hbase. storage.hostname=localhost # Zookeeper地址, 需要根据实际地址修改, 多台用','间隔. # 以下信息可参考, 不影响下面步骤. cache.db-cache = true cache.db-cache-clean-wait = 20 cache.db-cache-time = 180000 cache.db-cache-size = 0.5 # 以下信息可参考, 用于配置ElasticSearch索引信息, 不影响下面步骤. index.search.backend=elasticsearch # 索引方式. index.search.hostname=localhost # ElasticSearch主机地址, 多台用','间隔. index.search.port=9200 # ElasticSearch通信端口. index.search.elasticsearch.client-only=false
2. 在${TINKERPOP_HOME}/conf/目录创建janusgraph-gremlin-server.yaml, 添加内容如下:
host: 0.0.0.0
port: 8182
scriptEvaluationTimeout: 300000
channelizer: org.apache.tinkerpop.gremlin.server.channel.WebSocketChannelizer
graphs: {
graph: conf/janusgraph-hbase-server.properties
}
plugins:
- janusgraph.imports
scriptEngines: {
gremlin-groovy: {
imports: [java.lang.Math],
staticImports: [java.lang.Math.PI],
scripts: [scripts/empty-sample.groovy]}}
serializers:
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0, config: {ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { serializeResultToString: true }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}
processors:
- { className: org.apache.tinkerpop.gremlin.server.op.session.SessionOpProcessor, config: { sessionTimeout: 28800000 }}
- { className: org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor, config: { cacheExpirationTime: 600000, cacheMaxSize: 1000 }}
metrics: {
consoleReporter: {enabled: true, interval: 180000},
csvReporter: {enabled: true, interval: 180000, fileName: /tmp/gremlin-server-metrics.csv},
jmxReporter: {enabled: true},
slf4jReporter: {enabled: true, interval: 180000},
gangliaReporter: {enabled: false, interval: 180000, addressingMode: MULTICAST},
graphiteReporter: {enabled: false, interval: 180000}}
maxInitialLineLength: 4096
maxHeaderSize: 8192
maxChunkSize: 8192
maxContentLength: 65536
maxAccumulationBufferComponents: 1024
resultIterationBatchSize: 64
writeBufferLowWaterMark: 32768
writeBufferHighWaterMark: 65536
3. 在${TINKERPOP_HOME}/scripts/目录创建empty-sample.groovy,(此文件默认存在, 可直接使用.) 添加内容如下:
def globals = [:]
globals << [g : graph.traversal()]
六、运行Tinkerpop Server
[root@master apache-tinkerpop-gremlin-server-3.2.7]# bin/gremlin-server.sh conf/janusgraph-gremlin-server.yaml
1. 当显示如下内容时, 说明启动成功:
...... INFO] GremlinServer - Executing start up LifeCycleHook [INFO] Logger$info - Executed once at startup of Gremlin Server. [INFO] AbstractChannelizer - Configured application/vnd.gremlin-v1.0+gryo with org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0 [WARN] AbstractChannelizer - The org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0 serialization class is deprecated. [INFO] AbstractChannelizer - Configured application/vnd.gremlin-v1.0+gryo-lite with org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0 [INFO] AbstractChannelizer - Configured application/vnd.gremlin-v1.0+gryo-stringd with org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0 [INFO] AbstractChannelizer - Configured application/vnd.gremlin-v1.0+json with org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0 [INFO] AbstractChannelizer - Configured application/vnd.gremlin-v2.0+json with org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0 [INFO] AbstractChannelizer - Configured application/json with org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0 [INFO] GremlinServer$1 - Gremlin Server configured with worker thread pool of 1, gremlin pool of 32 and boss thread pool of 1. [INFO] GremlinServer$1 - Channel started at port 8182.
2. 当提示以下内容时, 修改文件, 去除\t
[ERROR] GremlinServer - Configuration file at conf/janusgraph-gremlin-server.yaml could not be found or parsed properly. [while scanning for the next token found character '\t(TAB)' that cannot start any token. (Do not use \t(TAB) for indentation) in 'reader', line 6, column 1: graph: conf/janusgraph-hbase-se ... ^ ]
七、通过Tinkerpop Console测试
1. 进入
apache-tinkerpop-gremlin-console-3.2.7
目录, 运行:
[root@master apache-tinkerpop-gremlin-console-3.2.7]# ./bin/gremlin.sh Feb 08, 2018 4:57:21 PM java.util.prefs.FileSystemPreferences$1 run INFO: Created user preferences directory. \,,,/ (o o) -----oOOo-(3)-oOOo----- plugin activated: tinkerpop.server plugin activated: tinkerpop.utilities plugin activated: tinkerpop.tinkergraph gremlin> :remote connect tinkerpop.server conf/remote.yaml ==>Configured localhost/127.0.0.1:8182 gremlin> :> g.V().count() ==> 0
运行无阻时, 说明部署完成. 可以继续往下进行.