概念

以下概念可以结合关系型数据库来理解

ES	mysql
索引库(index)	数据库
type	表
document	一条数据
field	字段
mapping映射	表结构ddl

索引库(index) ，document，field 对应Lucene中的相同概念，es底层基于lucene
在这里插入图片描述
分片 | 复制

高吞吐，高效率
加入一个索引有10亿个文档，占1t空间，一个集群中每个节点都不能消化这1T数据，这样把数据进行切分，例如5片，每一片放置到不同的节点中，可提供es的横向扩展能力；对一个搜索请求，将同时对不同节点上的每一片单独进行搜索，因为每一片对应的是lucene的一个索引库，都具备查询条件，增加了查询的效率。
高可用
es对每一片备份，默认是一份，对一个索引来说默认有10片，这10在实际物理存储时，会放在不同的节点中，复制与被复制片又会放置在不同的节点中，这样即使一个服务挂了，es仍然可以提供服务。

基本操作

通过postman和head插件基于es的restful接口进行操作

添加index1索引附带映射
PUT http://localhost:9200/index1

{
	"mappings":{
		"custom":{
			"properties":{
				"id":{
					"type":"long",//类型
					"store":true,//是否存储
					"index":"not_analyzed"//是否分析
				},
				"title":{
					"type":"text",
					"store":true,
					"index":"analyzed",
					"analyzer":"standard"
				},
				"content":{
					"type":"text",
					"store":true,
					"index":"analyzed",
					"analyzer":"standard"
				}
			}
		}
	}
}

向索引index1中添加名为custom的type 对应的mapping
post http://localhost:9200/index1/custom/_mappings

{
	"custom":{
		"properties":{
			"id":{
				"type":"long",
				"store":true,
				"index":"not_analyzed"
			},
			"title":{
				"type":"text",
				"store":true,
				"index":"analyzed",
				"analyzer":"standard"
			},
			"content":{
				"type":"text",
				"store":true,
				"index":"analyzed",
				"analyzer":"standard"
			}
		}
	}
}

向index1索引下的custom的type添加文档
post http://localhost:9200/index1/custom/1

{
"id":1,
"title":"这是一个文档标题",
"content":"断了的弦"
}

根据文档id查询
GET http://localhost:9200/index1/custom/1
termQuery进行查询
POST http://localhost:9200/index1/custom/_search
指定域为content

{
	"query":{
		"term":{
			"content":"断"
		}
	}
}

在这里插入图片描述
6. queryString进行查询
查询时首先对输入的内容进行分词 , 得到关键词 , 然后再根据关键词查询
POST http://localhost:9200/index1/custom/_search

{
	"query":{
		"query_string":{
			"default_field":"content",
			"query":"断了的弦该怎么连我的世界你已听不见"
		}
	}
}

在这里插入图片描述
使用head插件的基本查询可以获取到查询语句

查询分词器效果 , 关键词已经被小写

集群

更改config文件夹中的yml

http.cors.enabled: true
http.cors.allow-origin: "*"
#cluster.name可以确定你的集群名称,当你的elasticsearch集群在同一个网段中elasticsearch会自动的找到具有相同cluster.name的elasticsearch服务. 
# 所以当同一个网段具有多个elasticsearch集群时cluster.name就成为同一个集群的标识. 
cluster.name: my-elasticsearch
# 节点名称 不能出现相同的
node.name: node-1
#绑定监听IP 
network.host: 127.0.0.1

# 设置对外服务的http端口,默认为9200
http.port: 9201
# 设置节点间交互的tcp端口,默认是9300 
transport.tcp.port: 9301

# 这是一个集群中的主节点的初始列表,当节点(主节点或者数据节点)启动时使用这个列表进行探测 
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9301","127.0.0.1:9302"]

启动错误

access denied …IKAnalyzer.cfg.xml” “read
ailed - recovering as closed
java.security.AccessControlException: access denied (“java.io.FilePermission” “F
:\Program%20Files\ElasticSearch\elasticsearch-5.6.8\plugins\elasticsearch\config
\IKAnalyzer.cfg.xml” “read”)
可能是因为路径中出现了空格
https://blog.csdn.net/weixin_43043173/article/details/86769576
index索引名称不能有大写

对集群添加索引之后，观察分片位置，边框粗的为原片，对应的复制片都在不同的节点上
在这里插入图片描述
附测试用例


import org.elasticsearch.action.admin.indices.create.CreateIndexResponse;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.InetSocketTransportAddress;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentFactory;
import org.elasticsearch.index.query.*;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightField;
import org.elasticsearch.transport.client.PreBuiltTransportClient;
import org.junit.Before;
import org.junit.Test;

import java.io.IOException;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.Iterator;
import java.util.Map;

public class ESClientBasic {
    Settings settings;
    PreBuiltTransportClient preBuiltTransportClient;

    public static void main(String[] args) {
        //创建settings对象
        Settings settings = Settings.builder().put("cluster.name", "my-elasticsearch").build();
        //创建客户端对象 绑定连接的server
        PreBuiltTransportClient preBuiltTransportClient = new PreBuiltTransportClient(settings);
        try {
            preBuiltTransportClient.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("127.0.0.1"), 9301));
            preBuiltTransportClient.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("127.0.0.1"), 9302));
        } catch (UnknownHostException e) {
            e.printStackTrace();
        }
        //创建索引库
        CreateIndexResponse response = preBuiltTransportClient.admin().indices().prepareCreate("index_by_client3").get();
        String index = response.index();
        System.out.println(index);
        //关闭流
        preBuiltTransportClient.close();
    }

    @Before
    public void before() {
        //创建settings对象
        settings = Settings.builder().put("cluster.name", "my-elasticsearch").build();
        //创建客户端对象 绑定连接的server
        preBuiltTransportClient = new PreBuiltTransportClient(settings);
        try {
            preBuiltTransportClient.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("127.0.0.1"), 9301));
            preBuiltTransportClient.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("127.0.0.1"), 9302));
        } catch (UnknownHostException e) {
            e.printStackTrace();
        }
    }

    //添加文档
    @Test
    public void AddDocument() {
        XContentBuilder builder = null;
        for (int i = 2; i < 100; i++) {
            //创建document文档json
            try {
                builder = XContentFactory.jsonBuilder()
                        .startObject()
                        .field("id", i)
                        .field("title", "elasticsearch是一个基于lucene的搜索服务" + i)
                        .field("content", "我只能永远读者对白,读到我给你的伤害 ,我原谅不了我,就请你当作我已不在" + i)
                        .endObject();
                //创建索引库
                IndexResponse response = preBuiltTransportClient.prepareIndex()
                        .setIndex("index_by_client")
                        .setType("custom")
                        .setId(i + "").setSource(builder).get();
                System.out.println(response.status());
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
        //关闭流
        preBuiltTransportClient.close();
    }

    //指定文档id查询
    @Test
    public void searchById() {
        //创建查询对象 指定文档id
        QueryBuilder queryBuilder = QueryBuilders.idsQuery().addIds("1");
        executeQuery(queryBuilder);
    }

    //根据termquery查询
    @Test
    public void searchByTermQuery() {
        TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("content", "我");
        executeQuery(termQueryBuilder);
    }

    //根据queryString查询
    @Test
    public void searchByQueryString() {
//        QueryStringQueryBuilder queryStringQueryBuilder = QueryBuilders.queryStringQuery("5").defaultField("content");

//        QueryBuilder queryStringQueryBuilder = QueryBuilders.fuzzyQuery("content", "5");;
        //模糊查询
        QueryBuilder queryStringQueryBuilder = QueryBuilders.wildcardQuery("content", "*5");
        ;
        executeQuery(queryStringQueryBuilder);
    }

    //根据queryString查询
    @Test
    public void searchByQueryStringWithHighlight() {
//        QueryStringQueryBuilder queryStringQueryBuilder = QueryBuilders.queryStringQuery("5").defaultField("content");

//        QueryBuilder queryStringQueryBuilder = QueryBuilders.fuzzyQuery("content", "5");;
        //模糊查询
        QueryBuilder queryStringQueryBuilder = QueryBuilders.wildcardQuery("content", "*5");
        HighlightBuilder highlightBuilder = new HighlightBuilder();
        highlightBuilder.field("content");
        highlightBuilder.preTags("<h1>");
        highlightBuilder.postTags("</h1>");
        executeQueryWithHighLight(queryStringQueryBuilder, highlightBuilder);
    }

    private void executeQueryWithHighLight(QueryBuilder queryBuilder, HighlightBuilder highlightBuilder) {
        //执行查询 指定索引 , type
        SearchResponse response = preBuiltTransportClient.prepareSearch()
                .setIndices("index_by_client")
                .setTypes("custom")
                .setQuery(queryBuilder)
                .setFrom(0)
                .setSize(10)
                .highlighter(highlightBuilder)
                .get();
        SearchHits hits = response.getHits();
        System.out.println(hits.totalHits);

        Iterator<SearchHit> iterator = hits.iterator();
        while (iterator.hasNext()) {
            SearchHit searchHit = iterator.next();
            Map<String, Object> document = searchHit.getSource();
            System.out.println(document.get("title"));
            System.out.println(document.get("content"));
            System.out.println("-----------highlight result------");
            Map<String, HighlightField> highlightFields = searchHit.getHighlightFields();
            System.out.println(highlightFields);
            System.out.println("************************************************************");
        }
        preBuiltTransportClient.close();
    }

    private void executeQuery(QueryBuilder queryBuilder) {
        //执行查询 指定索引 , type
        SearchResponse response = preBuiltTransportClient.prepareSearch()
                .setIndices("index_by_client")
                .setTypes("custom")
                .setQuery(queryBuilder)
                .setFrom(0)
                .setSize(10)
                .get();
        SearchHits hits = response.getHits();
        System.out.println(hits.totalHits);

        Iterator<SearchHit> iterator = hits.iterator();
        while (iterator.hasNext()) {
            SearchHit searchHit = iterator.next();
            Map<String, Object> document = searchHit.getSource();
            System.out.println(document.get("title"));
            System.out.println(document.get("content"));
            System.out.println("----------------------------------------");
        }
        preBuiltTransportClient.close();
    }

}

SpringDataES

可能出现的问题

…ine 18 in XML document from class path resource [application.xml] is invalid…elasticsearch:transport-client的声明
这个可能是你的配置文件有问题, 打开编译后的配置文件, 看看哪里是否有报红, 我当时是在网上考了一份xml头, 但是里面有springmvc的约束, 但是我又没有引入springmvc的依赖, 导致解析失败, 编译后的配置文件中es标签头就会报异常, 删除mvc的约束就好了
在这里插入图片描述
对于索引库中已存在的索引不能再次创建

spring配置文件

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:context="http://www.springframework.org/schema/context"

       xmlns:elasticsearch="http://www.springframework.org/schema/data/elasticsearch"
       xsi:schemaLocation="
         http://www.springframework.org/schema/beans
         http://www.springframework.org/schema/beans/spring-beans-4.3.xsd
         http://www.springframework.org/schema/context
         http://www.springframework.org/schema/context/spring-context-4.3.xsd
         http://www.springframework.org/schema/data/elasticsearch
         http://www.springframework.org/schema/data/elasticsearch/spring-elasticsearch-1.0.xsd
       ">
    <context:component-scan base-package="com.rookie.es.repositories"/>
    <elasticsearch:transport-client id="esClient" cluster-name="my-elasticsearch"
                                    cluster-nodes="127.0.0.1:9301"/>
    <elasticsearch:repositories base-package="com.rookie.es.repositories"/>
    <bean name="elasticsearchTemplate"
          class="org.springframework.data.elasticsearch.core.ElasticsearchTemplate">
        <constructor-arg name="client" ref="esClient" />
    </bean>
</beans>

实体类


import org.springframework.data.annotation.Id;
import org.springframework.data.elasticsearch.annotations.Document;
import org.springframework.data.elasticsearch.annotations.Field;
import org.springframework.data.elasticsearch.annotations.FieldType;

@Document(indexName = "index_by_springdataes",type = "custom")
public class Article {
    @Id
    @Field(store = true,type = FieldType.Long)
    private long id;
    @Field(store = true,type = FieldType.text)
    private String title;
    @Field(store=true,type = FieldType.text)
    private String content;

    @Override
    public String toString() {
        return "Article{" +
                "id=" + id +
                ", title='" + title + '\'' +
                ", content='" + content + '\'' +
                '}';
    }

    public long getId() {
        return id;
    }

    public void setId(long id) {
        this.id = id;
    }

    public String getTitle() {
        return title;
    }

    public void setTitle(String title) {
        this.title = title;
    }

    public String getContent() {
        return content;
    }

    public void setContent(String content) {
        this.content = content;
    }
}

dao接口


import com.rookie.es.entity.Article;
import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;

public interface ArticleRepository extends ElasticsearchRepository<Article,Long> {

}

测试用例

import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.domain.PageRequest;
import org.springframework.data.elasticsearch.core.ElasticsearchTemplate;
import org.springframework.data.elasticsearch.core.query.NativeSearchQuery;
import org.springframework.data.elasticsearch.core.query.NativeSearchQueryBuilder;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;

import java.util.List;
import java.util.Optional;

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = "classpath:application.xml")
public class SpringDataESTest {
    @Autowired
    private ElasticsearchTemplate elasticsearchTemplate;
    @Autowired
    private ArticleRepository articleRepository;
    @Test
    public void creatIndex() {
        elasticsearchTemplate.createIndex(Article.class);
    }
    //当id已存在时,操作为更新
    @Test
    public void addDoc(){
        Article article = new Article();
        article.setId(4);
        article.setTitle("悟空");
//        article.setContent("乱世当空,恩怨休怀");
        article.setContent("跪一人为师,生死无关");
        articleRepository.save(article);
    }
    @Test
    public void delDoc(){
        Article article = new Article();
        article.setId(1);
        article.setTitle("悟空");
        article.setContent("乱世当空,恩怨休怀");
        articleRepository.delete(article);
    }
    @Test
    public void findAll(){
        Iterable<Article> articles = articleRepository.findAll();
        articles.forEach(article-> System.out.println(article));
    }
    @Test
    public void findById(){
        Optional<Article> optional = articleRepository.findById(1L);
        System.out.println(optional.isPresent()?optional.get():"不存在");
    }
    //通过springdata规范查询,方法名代表api
    @Test
    public void findByCustomCondition(){
//        List<Article> articles = articleRepository.findByTitle("悟空");

//        PageRequest pageRequest = PageRequest.of(0, 2);
//        List<Article> articles = articleRepository.findByTitleOrContent("悟空","跪一人为师,生死无关",pageRequest);

            //这种方式查询关键字和关键字之间属于and关系 因为没有一个content包含以下的所有关键字,所以没有记录
//        List<Article> articles = articleRepository.findByContent("叫一声佛祖跪一人为师,生死无关");
        //原生api查询关键字和关键字之间属于or关系
        NativeSearchQuery nativeSearchQuery = new NativeSearchQueryBuilder()
                .withQuery(QueryBuilders.queryStringQuery("叫一声佛祖跪一人为师,生死无关")
                        .defaultField("content"))
                .withPageable(PageRequest.of(0,2))
                .build();
        List<Article> articles = elasticsearchTemplate.queryForList(nativeSearchQuery, Article.class);
        articles.forEach(article-> System.out.println(article));
    }

}

挖坑记录~ElasticSearch基础

目录

概念

基本操作

集群

启动错误

SpringDataES

可能出现的问题

猜你喜欢