快学Big Data -- Elasticsearch 总结(二十六)

版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/xfg0218/article/details/82381700

Elasticsearch 总结

官网 : https://www.elastic.co

概述

    Elasticsearch是一个基于Apache Lucene(TM)的开源搜索引擎。无论在开源还是专有领域,Lucene可以被认为是迄今为止最先进、性能最好的、功能最全的搜索引擎库。

 

特点

 

Elasticsearch不仅仅是Lucene和全文搜索,我们还能这样去描述它:

1、分布式的实时文件存储,每个字段都被索引并可被搜索

2、分布式的实时分析搜索引擎

3、可以扩展到上百台服务器,处理PB级结构化或非结构化数据

Elasticsearch 安装

elasticsearch这是出于系统安全考虑设置的条件。由于ElasticSearch可以接收用户输入的脚本并且执行,为了系统安全考虑。

1-1)、安装

Elasticsearch 和elasticsearch-head-master下载:链接:http://pan.baidu.com/s/1czRe6y 密码:5q34 如果无法下载请联系作者。

 

[root@hadoop1elasticsearch]#wget https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.3.5/elasticsearch-2.3.5.tar.gz

 

[root@hadoop1 elasticsearch]# chmod a+x elasticsearch-2.1.1.tar.gz

[root@hadoop1 elasticsearch]# tar -zxvf elasticsearch-2.1.1.tar.gz

 

1-2)、创建用户与组

[root@hadoop1 bin]# groupadd elsearch

[root@hadoop1 bin]# useradd es  -g elsearch -p elasticsearch

[root@hadoop1 bin]# cd ../../

[root@hadoop1 elasticsearch]# chown -R es:elsearch  elasticsearch-2.1.1

[root@hadoop1 elasticsearch]# mkdir -p /path/to/data

[root@hadoop1 elasticsearch]# mkdir -p /path/to/logs

[root@hadoop1 opt]# cd /

[root@hadoop1 /]# chown -R es:elsearch  /path

1-3)、切换用户

[root@hadoop1 elasticsearch]# su es

[elsearch@hadoop1 elasticsearch-2.1.1]$ cd elasticsearch-2.1.1/

[elsearch@hadoop1 elasticsearch-2.1.1]$ cd bin/

1-4)、查看进程

[root@localhost nginx]# jps

53668 -- process information unavailable

53752 Jps

1-5)、安装Elasticsearch的插件

A)、Elasticsearch-head安装

[elsearch@hadoop1 bin]$ ./plugin install mobz/elasticsearch-head

-> Installing mobz/elasticsearch-head...

Trying https://github.com/mobz/elasticsearch-head/archive/master.zip ...

Downloading .........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................DONE

Verifying https://github.com/mobz/elasticsearch-head/archive/master.zip checksums if available ...

NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)

Installed head into /home/elasticsearch/elasticsearch-2.1.1/plugins/head

 

或者在elasticsearch的目录下创建以下目录,把https://github.com/mobz/elasticsearch-head/archive/master.zip下载后的压缩包解压到_site目录下即可,操作如下:

 

[root@hadoop1 elasticsearch-2.3.1]# mkdir -p plugins/head/

[root@hadoop1 elasticsearch-2.3.1]# unzip  elasticsearch-head-master.zip  -C /opt/elasticsearch-2.3.1/plugins/head/

[root@hadoop1 head]# cd elasticsearch-head-master

[root@hadoop1 elasticsearch-head-master]# mv * ../

[root@hadoop1 elasticsearch-head-master]# cd ../

[root@hadoop1 head]# rm -rf elasticsearch-head-master

[root@hadoop1 head]#  chmod -R es:elsearch *

 

B)、license 安装

[root@hadoop1 bin]# ./plugin install license

-> Installing license...

Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.3.1/license-2.3.1.zip ...

Downloading .......DONE

Verifying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.3.1/license-2.3.1.zip checksums if available ...

Downloading .DONE

Installed license into /usr/local/elasticsearch-2.3.1/plugins/license

 

C)、

1-6)、配置集群

[root@hadoop1 config]# vi elasticsearch.yml

4.修改配置

vi /bigdata/elasticsearch-2.3.1/config/elasticsearch.yml

#集群名称,通过组播的方式通信,通过名称判断属于哪个集群

cluster.name: bigdata

#节点名称,要唯一

node.name: es-1

#数据存放位置

path.data: /data/to/data

#日志存放位置

path.logs: /data/to/logs

#es绑定的ip地址

network.host: 172.16.0.14

# 端口

http.port: 9200

#初始化时可进行选举的节点

discovery.zen.ping.unicast.hosts: ["hadoop1",  "hadoop2",  "hadoop3"]

 

1-7)、传送到其他的机器上

[root@hadoop1 elasticsearch]# scp elasticsearch-2.1.1 hadoop2:$PWD

[root@hadoop1 elasticsearch]# scp elasticsearch-2.1.1 hadoop3:$PWD

 

1-8)、修改配置文件

[root@hadoop3 elasticsearch-2.3.1]# cd config/

[root@hadoop3 config]# vi  elasticsearch.yml

 

# ======================== Elasticsearch Configuration =========================

#

# NOTE: Elasticsearch comes with reasonable defaults for most settings.

#       Before you set out to tweak and tune the configuration, make sure you

#       understand what are you trying to accomplish and the consequences.

#

# The primary way of configuring a node is via this file. This template lists

# the most important settings you may want to configure for a production cluster.

#

# Please see the documentation for further information on configuration options:

# <http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html>

#

# ---------------------------------- Cluster -----------------------------------

#

# Use a descriptive name for your cluster:

#

  cluster.name: my-application

#

# ------------------------------------ Node ------------------------------------

#

# Use a descriptive name for the node:

#

  node.name: node-1

#

# Add custom attributes to the node:

#

# node.rack: r1

#

# ----------------------------------- Paths ------------------------------------

#

# Path to directory where to store the data (separate multiple locations by comma):

#

  path.data: /path/to/data

#

# Path to log files:

#

  path.logs: /path/to/logs

#

# ----------------------------------- Memory -----------------------------------

#

# Lock the memory on startup:

#

# bootstrap.mlockall: true

#

# Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory

# available on the system and that the owner of the process is allowed to use this limit.

#

# Elasticsearch performs poorly when the system is swapping the memory.

#

# ---------------------------------- Network -----------------------------------

#

# Set the bind address to a specific IP (IPv4 or IPv6):

#

  network.host: 192.168.215.134

#

# Set a custom port for HTTP:

#

  http.port: 9200

#

# For more information, see the documentation at:

# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html>

#

# --------------------------------- Discovery ----------------------------------

#

# Pass an initial list of hosts to perform discovery when new node is started:

# The default list of hosts is ["127.0.0.1", "[::1]"]

#

  discovery.zen.ping.unicast.hosts: ["hadoop1", "hadoop2", "hadoop3"]

#

# Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1):

#

# discovery.zen.minimum_master_nodes: 3

#

# For more information, see the documentation at:

# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html>

#

# ---------------------------------- Gateway -----------------------------------

#

# Block initial recovery after a full cluster restart until N nodes are started:

#

# gateway.recover_after_nodes: 3

#

# For more information, see the documentation at:

# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html>

#

# ---------------------------------- Various -----------------------------------

#

# Disable starting multiple nodes on a single system:

#

# node.max_local_storage_nodes: 1

#

# Require explicit names when deleting indices:

#

# action.destructive_requires_name: true

 

1-9)、运行Elasticsearch

A)、前台运行

[elsearch@hadoop1 bin]$ ./elasticsearch

[2016-08-27 14:18:15,854][INFO ][node                     ] [Trader] version[2.1.1], pid[7894], build[40e2c53/2015-12-15T13:05:55Z]

[2016-08-27 14:18:15,855][INFO ][node                     ] [Trader] initializing ...

[2016-08-27 14:18:16,743][INFO ][plugins                  ] [Trader] loaded [], sites []

[2016-08-27 14:18:16,993][INFO ][env                      ] [Trader] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [11gb], net total_space [17.6gb], spins? [unknown], types [rootfs]

[2016-08-27 14:18:22,613][INFO ][node                     ] [Trader] initialized

[2016-08-27 14:18:22,613][INFO ][node                     ] [Trader] starting ...

[2016-08-27 14:18:22,856][INFO ][transport                ] [Trader] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}, {[::1]:9300}

[2016-08-27 14:18:22,874][INFO ][discovery                ] [Trader] elasticsearch/HW-4e50uSkeSFEeNgEkMJQ

[2016-08-27 14:18:26,020][INFO ][cluster.service          ] [Trader] new_master {Trader}{HW-4e50uSkeSFEeNgEkMJQ}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)

[2016-08-27 14:18:26,242][INFO ][http                     ] [Trader] publish_address {127.0.0.1:9200}, bound_addresses {127.0.0.1:9200}, {[::1]:9200}

[2016-08-27 14:18:26,242][INFO ][node                     ] [Trader] started

[2016-08-27 14:18:26,257][INFO ][gateway                  ] [Trader] recovered [0] indices into cluster_state

^C[2016-08-27 14:21:54,274][INFO ][node                     ] [Trader] stopping ...

[2016-08-27 14:21:54,361][INFO ][node                     ] [Trader] stopped

[2016-08-27 14:21:54,362][INFO ][node                     ] [Trader] closing ...

[2016-08-27 14:21:54,371][INFO ][node                     ] [Trader] closed

B)、后端运行

[elsearch@hadoop1 bin]$ ./elasticsearch -d

 

C)、查看进程

bash-4.1$ jps

21177 Elasticsearch

21234 Jps

 

[es@hadoop2 bin]$ jps

14643 Jps

14606 Elasticsearch

 

 

[es@hadoop3 bin]$ jps

12896 Jps

12875 Elasticsearch

 

D)、查看界面

http://hadoop1:9200/_plugin/head/

 

 

 

从中可以看出node-2是主节点,其他的副节点,其中0,1,2,3,4是leader其他的都是flowller是同步数据的。

 

1-10)、停止ES

若是要停止ES服务,则输入sh elasticsearch stop,则输出如下图,则ES成功停止。

kill `ps -ef | grep Elasticsearch | grep -v grep | awk '{print $2}'`

 

1-11)、查看Zookeeper保存的信息

1-12)、一键启动脚本

请用es用户执行以下脚本,注意脚本的权限

[es@hadoop1 start-single]$ ls -l

-rwxr-xr-x. 1  es   elsearch     166 Apr 23 11:14 start-elasticsearch.sh

 

 

执行脚本

[es@hadoop1 start-single]$ sh start-elasticsearch.sh

cat /opt/start-script/slave |while read line

do

{

echo "$line"

ssh $line "source /etc/profile;su es;cd /opt/elasticsearch-2.3.1/bin;sh elasticsearch -d"

}&

wait

Done

 

Curl  常用操作

1-1)、检测集群是否健康

[es@hadoop1 root]$ curl 'hadoop1:9200/_cat/health?v'

 

epoch      timestamp cluster        status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent

1493390489 22:41:29  my-application green           3         3     12   6    0    0        0             0                  -                100.0%

1-2)、查看集群列表

[es@hadoop1 root]$ curl 'hadoop1:9200/_cat/nodes?v'

 

host            ip              heap.percent ram.percent load node.role master name   

192.168.132.148 192.168.132.148            5          40 0.00 d         m      node-  

192.168.132.139 192.168.132.139            2          38 0.00 d         m      node-3

192.168.132.140 192.168.132.140            5          35 0.00 d         *      node-1

1-3)、查看所有的索引

[es@hadoop1 root]$ curl 'hadoop1:9200/_cat/indices?v'

 

health status index    pri rep docs.count docs.deleted store.size pri.store.size

green  open   .kibana    1   1          1            0      6.3kb          3.1kb

green  open   customer   5   1         45            0    117.7kb         58.8kb

1-4)、创建索引

[es@hadoop1 root]$ curl -XPUT 'hadoop1:9200/customer?pretty'

会看到以下信息

{

  "acknowledged" : true

}

1-5)、插入数据

[es@hadoop1 root]$ curl -XPUT 'hadoop1:9200/customer/external/1?pretty' -d '{"name":"xiaozhang","age":"18"}'

 

回车可以看到以下信息

 

{

  "_index" : "customer",

  "_type" : "external",

  "_id" : "1",

  "_version" : 1,

  "_shards" : {

    "total" : 2,

    "successful" : 2,

    "failed" : 0

  },

  "created" : true

}

 

 

1-6)、获取数据

[es@hadoop1 root]$ curl -XGET 'hadoop1:9200/customer/external/1?pretty'

 

回车即可看到以下信息

 

{

  "_index" : "customer",

  "_type" : "external",

  "_id" : "1",

  "_version" : 1,

  "found" : true,

  "_source" : {

    "name" : "xiaozhang",

    "age" : "18"

  }

}

1-7)、删除索引

 

[es@hadoop1 root]$ curl -XDELETE 'hadoop1:9200/customer?pretty'

{

  "acknowledged" : true

}

 

1-8)、修改数据

先查看id为3的数据

 

[es@hadoop1 root]$ curl -XPUT 'hadoop1:9200/customer/external/3?pretty' -d '{"name":"xiaoxu","age":"23"}'

 

 

1-9)、更新数据

查看修改ID为1的数据

 

[es@hadoop1 root]$ curl -XPOST 'hadoop1:9200/customer/external/1/_update?pretty' -d '{"doc":{"name":"Jane Han"}}'

 

1-10)、删除数据

删除ID为1的数据

[es@hadoop1 root]$ curl -XDELETE 'hadoop1:9200/customer/external/1?pretty'

1-11)、批处理数据

JSON文件下载地址:链接:http://pan.baidu.com/s/1bLZqn8 密码:m6he

[es@hadoop1 testData]$ curl -XPOST 'hadoop1:9200/bank/account/_bulk?pretty' --data-binary "@/opt/testData/accounts.json"

 

 

 

1-12)、常见错误处理

问题描述

elasticsearch6.x {"error":"Content-Type header [application/x-www-form-urlencoded] is not supported" }

 

问题解答

[es@hadoop1 root]$ curl -H "Content-Type: application/json" -XDELETE 'hadoop1:9200/customer/external/1?pretty'

代码实例

在项目中导入项目中/elasticsearch-2.3.1/lib

1-1)、实体类

package com.sdcet;

 

public class Medicine {

private Integer id;

private String name;

private String function;

 

public Integer getId() {

return id;

}

 

public void setId(Integer id) {

this.id = id;

}

 

public String getName() {

return name;

}

 

public void setName(String name) {

this.name = name;

}

 

public String getFunction() {

return function;

}

 

public void setFunction(String function) {

this.function = function;

}

 

public Medicine() {

}

 

public Medicine(Integer id, String name, String function) {

this.id = id;

this.name = name;

this.function = function;

}

 

@Override

public String toString() {

return "Medicine [id=" + id + ", name=" + name + ", function="

+ function + "]";

}

 

}

 

1-2)、工具类

package com.sdcet;

 

import java.io.IOException;

import org.elasticsearch.common.xcontent.XContentBuilder;

import org.elasticsearch.common.xcontent.XContentFactory;

 

public class JsonUtil {

 

/**

 * 实现将实体对象转换成json对象

 *

 * @param medicine

 *            Medicine对象

 * @return

 */

public static String obj2JsonData(Medicine medicine) {

String jsonData = null;

try {

// 使用XContentBuilder创建json数据

XContentBuilder jsonBuild = XContentFactory.jsonBuilder();

jsonBuild.startObject().field("id", medicine.getId())

.field("name", medicine.getName())

.field("funciton", medicine.getFunction()).endObject();

jsonData = jsonBuild.string();

System.out.println(jsonData);

} catch (IOException e) {

e.printStackTrace();

}

return jsonData;

}

}

 

1-3)、工厂类

package com.sdcet;

 

import java.util.ArrayList;

import java.util.List;

 

public class DataFactory {

public static DataFactory dataFactory = new DataFactory();

 

private DataFactory() {

 

}

 

public DataFactory getInstance() {

return dataFactory;

}

 

public static List<String> getInitJsonData() {

List<String> list = new ArrayList<String>();

for (int i = 0; i < 10; i++) {

String data1 = JsonUtil.obj2JsonData(new Medicine(i, "银花 感冒 颗粒",

"功能主治:银花感冒颗粒 ,头痛,清热,解表,利咽。"));

list.add(data1);

}

return list;

}

}

 

1-4)、主类

package com.sdcet;

 

import java.net.InetAddress;

import java.net.UnknownHostException;

import java.util.ArrayList;

import java.util.List;

 

import org.elasticsearch.action.index.IndexRequestBuilder;

import org.elasticsearch.action.index.IndexResponse;

import org.elasticsearch.action.search.SearchResponse;

import org.elasticsearch.client.Client;

import org.elasticsearch.client.transport.TransportClient;

import org.elasticsearch.common.transport.InetSocketTransportAddress;

import org.elasticsearch.index.query.QueryBuilder;

import org.elasticsearch.index.query.QueryBuilders;

import org.elasticsearch.search.SearchHit;

import org.elasticsearch.search.SearchHits;

 

public class ElasticSearchHandler {

private Client client;

 

public ElasticSearchHandler() {

// 使用本机做为节点

this("127.0.0.1");

}

 

public ElasticSearchHandler(String ipAddress) {

try {

client = TransportClient

.builder()

.build()

.addTransportAddress(

new InetSocketTransportAddress(InetAddress

.getByName("127.0.0.1"), 9300));

} catch (UnknownHostException e) {

e.printStackTrace();

}

// 集群连接超时设置

/*

 * Settings settings =

 * ImmutableSettings.settingsBuilder().put("client.transport.ping_timeout"

 * , "10s").build(); client = new TransportClient(settings);

 */

// client = new TransportClient()

// .addTransportAddress(new InetSocketTransportAddress(ipAddress,

// 9200));

 

}

 

/**

 * 建立索引,索引建立好之后,会在elasticsearch-0.20.6\data\elasticsearch\nodes\0创建所以你看

 *

 * @param indexName

 *            为索引库名,一个es集群中可以有多个索引库。 名称必须为小写

 * @param indexType

 *            Type为索引类型,是用来区分同索引库下不同类型的数据的,一个索引库下可以有多个索引类型。

 * @param jsondata

 *            json格式的数据集合

 *

 * @return

 */

public void createIndexResponse(String indexname, String type,

List<String> jsondata) {

// 创建索引库 需要注意的是.setRefresh(true)这里一定要设置,否则第一次建立索引查找不到数据

IndexRequestBuilder requestBuilder = client.prepareIndex(indexname,

type).setRefresh(true);

for (int i = 0; i < jsondata.size(); i++) {

System.out.println(jsondata.get(i) + "-");

requestBuilder.setSource(jsondata.get(i)).execute().actionGet();

}

 

}

 

/**

 * 创建索引

 *

 * @param client

 * @param jsondata

 * @return

 */

public IndexResponse createIndexResponse(String indexname, String type,

String jsondata) {

IndexResponse response = client.prepareIndex(indexname, type)

.setSource(jsondata).execute().actionGet();

return response;

}

 

/**

 * 执行搜索

 *

 * @param queryBuilder

 * @param indexname

 * @param type

 * @return

 */

public List<Medicine> searcher(QueryBuilder queryBuilder, String indexname,

String type) {

List<Medicine> list = new ArrayList<Medicine>();

SearchResponse searchResponse = client.prepareSearch(indexname)

.setTypes(type).setQuery(queryBuilder).execute().actionGet();

// SearchHits hits = searchResponse.hits();

SearchHits hits = searchResponse.getHits();

System.out.println("查询到记录数=" + hits.getTotalHits());

SearchHit[] searchHists = hits.getHits();

if (searchHists.length > 0) {

for (SearchHit hit : searchHists) {

Integer id = (Integer) hit.getSource().get("id");

String name = (String) hit.getSource().get("name");

String function = (String) hit.getSource().get("funciton");

list.add(new Medicine(id, name, function));

}

}

return list;

}

 

public static void main(String[] args) {

ElasticSearchHandler esHandler = new ElasticSearchHandler();

List<String> jsondata = DataFactory.getInitJsonData();

String indexname = "indexdemo";

String type = "typedemo";

// 创建索引

// esHandler.createIndexResponse(indexname, type, jsondata);

// 查询条件F

// QueryBuilder queryBuilder = QueryBuilders.fuzzyQuery("name", "感冒");

QueryBuilder queryBuilder = QueryBuilders.functionScoreQuery();

// QueryBuilder queryBuilder = QueryBuilders.fieldQuery("name", "感冒");

// QueryBuilders.se

/*

 * QueryBuilder queryBuilder = QueryBuilders.boolQuery()

 * .must(QueryBuilders.termQuery("id", 1));

 */

List<Medicine> result = esHandler.searcher(queryBuilder, indexname,

type);

for (int i = 0; i < result.size(); i++) {

Medicine medicine = result.get(i);

System.out.println("(" + medicine.getId() + ")药品名称:"

+ medicine.getName() + "\t\t" + medicine.getFunction());

}

}

}

 

 

查看效果

查看执行时间以及条数,感觉还可以

 

 

 

 

 

查看保存的数据结构

 

 

 

 

 

注意以上数据保存的路径以及数据的后缀。还有一个后缀为lock的文件,是咋写入数据时的对数据的处理的进度。

 

查询语句

must: 文档必须完全匹配条件
should: should下面会带一个以上的条件,至少满足一个条件,这个文档就符合shouldmust_not: 文档必须不匹配条件

详细请见http://www.cnblogs.com/yjf512/p/4897294.html

1-1)、精确查找

{

  "query": {

    "match_phrase": {

      "name": "小青"

    }

  }

}

 

1-2)、模糊查找

{

  "query": {

    "match_phrase": {

      "name": "小青"

    }

  }

}

 

 

1-3)、查找多个字段的数据

{

  "query": {

    "multi_match": {

      "query": "小张感冒",

      "fields": [

        "name",

        "funciton"

      ]

    }

  }

}

1-4)、我们希望完全匹配的文档占的评分比较高,则需要使用best_fields

 

{

  "query": {

    "multi_match": {

      "query": "小张感冒",

      "type": "best_fields",

      "fields": [

        "name",

        "funciton"

      ],

      "tie_breaker": 0.1

    }

  }

}

 

意思就是完全匹配"宝马 发动机"的文档评分会比较靠前,如果只匹配宝马的文档评分乘以0.3的系数

 

 

1-5)、多字段匹配

  

{

  "query": {

    "multi_match": {

      "query": "小张感冒",

      "type": "most_fields",

      "fields": [

        "name",

        "funciton"

      ],

      "tie_breaker": 0.1

    }

  }

}

 

1-6)、我们会希望这个词条的分词词汇是分配到不同字段中的,那么就使用cross_fields

{

  "query": {

    "term": {

      "content": "1"

    }

  }

}

 

 

1-7)、高亮设置

 

{

  "query": {

    "match_phrase": {

      "name": "大青叶"

    }

  },

  "highlight": {

    "fields": {

      "name": {}

    }

  }

}

 

 

1-8)、URL 查询

http://localhost:9200/indexdemo/typedemo/AVbVXZ5YQST3amZyeBjV?_source=id,name

 

 

Elasticsearch 数据映射给Hive

1-1)、下载相对应的JAR

链接:http://pan.baidu.com/s/1bp9vU2N 密码:1nbz

1-2)、加载JAR

hive > ADD JAR /home/xiaoxu/elasticsearch-hadoop-2.3.4.jar;

1-3)、映射数据

hive > create EXTERNAL  table es_hive_sp_address(

regtime string,

uid int,

mobile string,

username string 

)

STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'

TBLPROPERTIES('es.resource' = 'sp_address/sp_address', 'es.nodes'='192.168.31.122', 'es.port'='9003','es.nodes.wan.only'='true');

1-4)、查看数据

hive > select * from es_hive_sp_address;

1 01 110000 北京市

2 01 120000 天津市

3 01 130000 河北省

4 01 140000 山西省

5 01 150000 内蒙古自治区

6 01 210000 辽宁省

7 01 220000 吉林省

8 01 230000 黑龙江省

9 01 310000 上海市

***************************

elasticsearch-sql使用

github介绍:https://github.com/NLPchina/elasticsearch-sql

1-1)、软件安装

$ ./plugin install https://github.com/NLPchina/elasticsearch-sql/releases/download/2.3.1.1/elasticsearch-sql-2.3.1.1.zip

1-2)、软件使用

http://192.168.31.47:9200/_plugin/sql/

1-3)、API 调用elasticsearch-sql

public void testJDBC() throws Exception {

        Properties properties = new Properties();

        properties.put("url", "jdbc:elasticsearch://192.168.31.47:9200/" + TestsConstants.TEST_INDEX);

        DruidDataSource dds = (DruidDataSource) ElasticSearchDruidDataSourceFactory.createDataSource(properties);

        Connection connection = dds.getConnection();

        PreparedStatement ps = connection.prepareStatement("SELECT  gender,lastname,age from  " + TestsConstants.TEST_INDEX + " where lastname='Heath'");

        ResultSet resultSet = ps.executeQuery();

        List<String> result = new ArrayList<String>();

        while (resultSet.next()) {

              System.out.println(resultSet.getString("lastname") + "," + resultSet.getInt("age") + "," + resultSet.getString("gender"))

        }

        ps.close();

        connection.close();

        dds.close();

    }

猜你喜欢

转载自blog.csdn.net/xfg0218/article/details/82381700