This paper briefly record a practice course, involves presto-mysql, presto-elasticsearch, and related inquiry
1 Download and install presto-0.228
<1> Download
Server
https://repo1.maven.org/maven2/com/facebook/presto/presto-server/0.228/presto-server-0.228.tar.gz
Client
https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.228/presto-cli-0.228-executable.jar
Related jar
https://repo1.maven.org/maven2/com/facebook/presto/presto-base-jdbc/0.228/presto-base-jdbc-0.228.jar
https://repo1.maven.org/maven2/com/facebook/presto/presto-spi/0.228/presto-spi-0.228.jar
The official document:
http://prestodb.github.io/docs/current/connector/elasticsearch.html
<2> the installation:
1> unzip
tar -zxvf presto-server-0.228.tar.gz
2> Create a configuration directory etc etc / catalog
cd soon-server-0228 /
mk you etc
mkdir etc
mkdir data
cd etc
mkdir catalog
3> Creating a cluster configuration config.properties
Under the new vim config.properties etc directory
document content:
# Whether Coordinator scheduling node
coordinator=true
# Whether as work. For large clusters, the worker to do the work in the coordinator will affect query performance
node-scheduler.include-coordinator=true
# Specify the HTTP port. Presto uses HTTP to communicate with the external and internal
http-server.http.port=9080
# Can be used to query the maximum total memory
query.max-memory=8GB
# Can be used to query the largest single junction memory
query.max-memory-per-node=1GB
query.max-total-memory-per-node=2GB
#Presto using Discovery Service to find all the nodes in the cluster. Each instance will be registered in the Presto service in Discovery at startup. This simplifies deployment, no additional service, Presto's coordinator built a Discovery service. Also uses HTTP port
discovery-server.enabled=true
The domain #Coordinator nodes or IP, Presto using Discovery Service to find all the nodes in the cluster. Each instance will be registered in the Presto service in Discovery at startup. This simplifies deployment, no additional service, Presto's coordinator built a Discovery service. Also uses HTTP port
discovery.uri=http://xinyi:9080
4> Create a runtime configuration jvm.config
Under the new etc directory
vim jvm.config
-server
-Xmx16G
-XX:+UseG1GC
-XX:G1HeapRegionSize=32M
-XX:+UseGCOverheadLimit
-XX:+ExplicitGCInvokesConcurrent
-XX:+HeapDumpOnOutOfMemoryError
-XX:+ExitOnOutOfMemoryError
5> Create a log configuration log.properties
Under the new etc directory
vim log.properties
com.facebook.presto = ABOUT
6> Create node.properties, node configuration
Under the new etc directory vim log.properties
document content:
# Custom environment name, the name node environment Presto cluster must be the same.
node.environment=production
# Unique identifier identifies each node must be as one. Even if restart or upgrade Presto must also keep the original logo.
node.id=ffffffff-ffff-ffff-ffff-fffffffffff1
# Data Directory, Presto use it to save the log and other data
node.data-dir=/opt/presto-server-0.228/data
7> Start
cd / installation directory / bin
./launcher start back to start
./launcher stop stop
./launcher run reception starts, the output log
./launcher restart restart
8> JDK Configuration
The installation directory by modifying presto: / presto / bin / launther
Modify launther, introduced jdk1.8.
vim launther
PATH = / jdk installation directory / bin: $ PATH
exec "$(dirname "$0)/launcher.py" "$@"
9> / installation directory / var / log log file to find the following location
launcher.log
server.log
http-request.log
10> View web interface http: // serverIp: 9080 / ui /
<3> Client Installation
Rename client jar package presto-cli-0.228-executable.jar is executable presto
mv presto-cli-0.228-executable.jar presto
Use the command:
./presto --server locahost:9080 --catalog mysql--schema test
2 early-mysql
<1> Create mysql.properties in the / etc / catalog / directory
connector.name=mysql
connection-url=jdbc:mysql://localhost:3306
connection-user=root
connection-password=root
<2> Restart presto-server
/launcher restart
<3> test
Executed in the client installation directory
./presto --server locahost:9080 --catalog mysql--schema test
presto:es> select * from mysql.test.test;
3 early-Elasticsearch
<1> Create elasticsearch.properties in the / installation directory / etc / catalog / directory
connector.name=elasticsearch
#elasticsearch.default-schema=default
# Specify the directory where the table definition file
elasticsearch.table-description-directory=etc/elasticsearch/
elasticsearch.scroll-size=1000
elasticsearch.scroll-timeout=2s
elasticsearch.request-timeout=2s
elasticsearch.max-request-retries=5
elasticsearch.max-request-retry-time=10s
<2> Create elasticsearch directory
cd / installation directory / etc /
mkdir elasticsearch
cd elasticsearch
<3> description file definition table (custom table .json)
Each mapping table associating json file name definition table, ES address, index name, type, format field corresponding to
cd elasticsearch
vim test.json
{"tableName": "es_test",
"schemaName": "es",
"host": "es-ip地址",
"port": 9300,
"clusterName": "my-application",
"index": "test",
"indexExactMatch": false,
"type": "test",
"columns": [
{
"name": "name",
"type": "varchar",
"jsonPath":"name",
"jsonType":"varchar"
},
{
"name": "age",
"type": "integer",
"jsonPath":"age",
"jsonType":"integer"
}
]
}
<4> Restart presto-server
/launcher restart
<5> test
./presto --server locahost:9080 --catalog elasticsearch --schema es
presto:es> select * from elasticsearch.es.es_test;
name | age
------+-----
HL | 12
HLl | 18
(2 rows)
4 Multi-source data query
./presto --server locahost:9080 --catalog elasticsearch --schema es
presto:es> select * from mysql.test.user t left join elasticsearch.es.es_test t1 on t.age=t1.age;
5 JDBC-JAVA
<dependency>
<groupId>com.facebook.presto</groupId>
<artifactId>presto-jdbc</artifactId>
<version>0.228</version>
</dependency>
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.Statement;
public class PrestoJdbcDemo {
public static void main(String[] args) throws Exception{
Class.forName("com.facebook.presto.jdbc.PrestoDriver");
Connection connection = DriverManager.getConnection("jdbc:presto://localhost:9080/mysql/test","root",null); ;
Statement stmt = connection.createStatement();
ResultSet rs = stmt.executeQuery("select * from mysql.test.user t left join elasticsearch.es.es_test t1 on t.age=t1.age");
while (rs.next()) {
System.out.println(rs.getString(1));
}
rs.close();
connection.close();
}
}