分布式全文搜索引擎——Elasticsearch

其他 2019-06-29 10:38:19 阅读次数: 0

1.安装Elasticsearch

　　a.下载：从官网下载 Elasticsearch，地址：https://www.elastic.co/cn/downloads/past-releases#elasticsearch

　　b.启动：解压后，在 bin 目录打开 elasticsearch.bat 启动服务

　　c.访问：在浏览器中访问 http://127.0.0.1:9200/ 检查是否安装成功

　　　　注意：端口9300：Java程序访问的端口；端口9200：浏览器访问的端口

　　　　　　　Elasticsearch 的版本需要跟后面的 Spring-Data-Elasticsearch 的版本相符，不然会不兼容（Spring 5.1.1 对应 Elasticsearch 6.8.0）

2.安装Head管理Elasticsearch插件

　　a.安装nodejs：在nodejs官网下载msi，地址：https://nodejs.org/en/download/，安装后在CMD命令行输入 node -v 查看是否安装成功

　　b.安装grunt：在CMD命令行输入 npm install -g grunt-cli 安装grunt，安装后输入 grunt -version 查看是否安装成功

　　c.下载Head：在GitHub下载并解压 elasticsearch-head-master，地址：https://github.com/mobz/elasticsearch-head

　　d.修改Head配置：修改Head根目录下Gruntfile.js，添加：hostname:'*',

　　

　　e.修改Elasticsearch配置：修改 Elasticsearch 的 config 目录下的 elasticsearch.yml 文件，添加如下配置。并重启

network.host: 127.0.0.1
# 解决elasticsearch-head 集群健康值: 未连接问题
http.cors.enabled: true 
http.cors.allow-origin: "*"

　　f.安装Head：在 elasticsearch-head-master 根目录下执行命令 npm install 进行安装

　　g.启动Head：同样在 elasticsearch-head-master 根目录下执行命令 grunt server（或者npm run start）

　　h.访问Head：在浏览器中访问 http://localhost:9100/ 即可进入管理界面

3.安装IK中文分词器

　　a.下载：在GitHub上下载对应版本的IK分词器zip包，地址：https://github.com/medcl/elasticsearch-analysis-ik/releases

　　b.注意：Elasticsearch和IK分词器必须版本统一

　　c.解压安装：解压到 elasticsearch 的 plugins 目录下，并改名为 ik。并重启 elasticsearch

　　

4.使用 Spring-Data-Elasticsearch 操作 Elasticsearch

　　a.导入pom依赖

 <properties>

    ......

    <!-- spring -->
    <spring.version>5.1.1.RELEASE</spring.version>

  </properties>

  <dependencies>
    <!-- elasticsearch -->
    <dependency>
      <groupId>org.springframework.data</groupId>
      <artifactId>spring-data-elasticsearch</artifactId>
      <version>3.1.1.RELEASE</version>
    </dependency>

    <!-- spring-5.X -->
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-core</artifactId>
      <version>${spring.version}</version>
    </dependency>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-webmvc</artifactId>
      <version>${spring.version}</version>
    </dependency>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-oxm</artifactId>
      <version>${spring.version}</version>
    </dependency>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-tx</artifactId>
      <version>${spring.version}</version>
    </dependency>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-aop</artifactId>
      <version>${spring.version}</version>
    </dependency>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-context-support</artifactId>
      <version>${spring.version}</version>
    </dependency>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-test</artifactId>
      <version>${spring.version}</version>
    </dependency>

    <!-- AOP-AspectJ spring-aop依赖 -->
    <dependency>
      <groupId>org.aspectj</groupId>
      <artifactId>aspectjrt</artifactId>
      <version>1.8.6</version>
    </dependency>
    <dependency>
      <groupId>org.aspectj</groupId>
      <artifactId>aspectjweaver</artifactId>
      <version>1.8.6</version>
    </dependency>

    <!-- jackson-json spring-mvc依赖 -->
    <dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-databind</artifactId>
      <version>2.9.4</version>
    </dependency>
    <dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-core</artifactId>
      <version>2.9.4</version>
    </dependency>
    <dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-annotations</artifactId>
      <version>2.9.4</version>
    </dependency>

    <!-- fastjson -->
    <dependency>
      <groupId>com.alibaba</groupId>
      <artifactId>fastjson</artifactId>
      <version>1.2.47</version>
    </dependency>

    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.11</version>
      <scope>test</scope>
    </dependency>
  </dependencies>

　　b.创建BaseEntity基础实体类

public class BaseEntity {

    @Id
    private String id;

    public String getId() {
        return id;
    }

    public void setId(String id) {
        this.id = id;
    }

}

 　　c.创建Article和Author实体类

//index相当于数据库，type相当于表
@Document(indexName = "elasticsearch", type = "article")
public class Article extends BaseEntity {

    private String title;
    private String content;
    //内嵌对象
    @Field(type = FieldType.Nested)
    private List<Author> authors;

    public String getTitle() {
        return title;
    }

    public void setTitle(String title) {
        this.title = title;
    }

    public List<Author> getAuthors() {
        return authors;
    }

    public void setAuthors(List<Author> authors) {
        this.authors = authors;
    }

    public String getContent() {
        return content;
    }

    public void setContent(String content) {
        this.content = content;
    }

    @Override
    public String toString() {
        return "Article: id[" + this.getId() + "], title[" + title + "], content[" + content + "], authors[" + authors + "]";
    }
}
public class Author {

    private String name;

    private int year;

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public int getYear() {
        return year;
    }

    public void setYear(int year) {
        this.year = year;
    }

    @Override
    public String toString() {
        return "Author: name[" + name + "], year[" + year + "]";
    }
}

　　d.创建BaseDao基础DAO

public class BaseDao<T extends BaseEntity> {

    @Resource(name="elasticsearchTemplate")
    protected ElasticsearchTemplate esTemplate;
    protected Class<T> clazz;

    @PostConstruct
    private void construct(){
        clazz = (Class)((ParameterizedType)this.getClass().getGenericSuperclass()).getActualTypeArguments()[0];
    }

    /**
     * 保存单个对象
     * @param t 对象
     * @return
     */
    public boolean save(T t){
        String id = t.getId();
        if (id == null) {
            id = UUID.randomUUID().toString().replaceAll("-", "");
            t.setId(id);
        }
        IndexQuery indexQuery = new IndexQueryBuilder().withId(id).withObject(t).build();
        esTemplate.index(indexQuery);
        return true;
    }

    /**
     * 保存多个
     * @param list 对象集合
     * @return
     */
    public boolean save(List<T> list){
        List<IndexQuery> queries = new ArrayList<IndexQuery>();
        for (T t : list) {
            String id = t.getId();
            if (id == null) {
                id = UUID.randomUUID().toString().replaceAll("-", "");
                t.setId(id);
            }
            IndexQuery indexQuery = new IndexQueryBuilder().withId(id).withObject(t).build();
            queries.add(indexQuery);
        }
        esTemplate.bulkIndex(queries);
        return true;
    }

    /**
     * 根据ID删除
     * @param id 对象ID
     * @return
     */
    public boolean deleteById(String id){
        esTemplate.delete(clazz, id);
        return true;
    }

    /**
     * 根据多个ID删除
     * @param idList 对象ID集合
     * @return
     */
    public boolean deleteByIds(List<String> idList) {
        DeleteQuery deleteQuery = new DeleteQuery();
        BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery();
        if(idList != null) {
            for (String id : idList) {
                queryBuilder.should(QueryBuilders.matchQuery("id", id));
            }
        }
        deleteQuery.setQuery(queryBuilder);;
        esTemplate.delete(deleteQuery, clazz);
        return true;
    }

    /**
     * 根据过滤条件删除
     * @param filter 过滤条件Map
     * @return
     */
    public boolean delete(Map<String,Object> filter){
        DeleteQuery deleteQuery = new DeleteQuery();
        BoolQueryBuilder queryBuilder = QueryBuilders. boolQuery();
        if(filter != null) {
            for (Map.Entry<String, Object> entry : filter.entrySet()) {
                //"content.keyword" 表示不进行分词（精准查询），解决 "content" 分词后 term 查不到数据的问题
                queryBuilder.must(QueryBuilders.matchQuery(entry.getKey() + ".keyword", entry.getValue()));
            }
        }
        deleteQuery.setQuery(queryBuilder);
        esTemplate.delete(deleteQuery, clazz);
        return true;
    }

    /**
     * 根据条件查询集合
     * @param filter 过滤条件Map
     * @param highFields 高亮字段
     * @param sortField 排序字段
     * @param order 正序倒序
     * @return
     */
    public List<T> queryList(Map<String, Object> filter, final List<String> highFields, String sortField, SortOrder order) {
        NativeSearchQueryBuilder searchBuilder = new NativeSearchQueryBuilder();

        //-----------------------------高亮-----------------------------
        List<Field> fieldList = new ArrayList<>();
        if(highFields != null) {
            for (String highField : highFields) {
                fieldList.add(new HighlightBuilder.Field(highField).preTags("<em>").postTags("</em>").fragmentSize(250));
            }
        }
        searchBuilder.withHighlightFields(fieldList.toArray(new Field[fieldList.size()]));

        //-----------------------------排序-----------------------------
        if (sortField != null && order != null){
            searchBuilder.withSort(new FieldSortBuilder(sortField + ".keyword").order(order));
        }

        //-----------------------------过滤条件-----------------------------
        BoolQueryBuilder queryBuilder= QueryBuilders.boolQuery();
        for (Map.Entry<String, Object> entry : filter.entrySet()) {
            queryBuilder.must(QueryBuilders.matchQuery(entry.getKey(), entry.getValue()));
        }
        searchBuilder.withQuery(queryBuilder);

        //-----------------------------查询建立-----------------------------
        SearchQuery searchQuery = searchBuilder.build();
        Page<T> page = null;
        //如果设置高亮
        if (highFields != null && highFields.size() > 0) {
            page = esTemplate.queryForPage(searchQuery, clazz, new SearchResultMapper() {
                @Override
                public <T> AggregatedPage<T> mapResults(SearchResponse response,Class<T> clazz, Pageable pageable) {
                    List<T> list = new ArrayList<T>();
                    for (SearchHit searchHit : response.getHits()) {
                        if (response.getHits().getHits().length <= 0) {
                            return null;
                        }
                        Map<String, Object> entityMap = searchHit.getSourceAsMap();
                        for (String highName : highFields) {
                            String highValue = searchHit.getHighlightFields().get(highName).fragments()[0].toString();
                            entityMap.put(highName, highValue);
                        }
                        T t = JSONObject.parseObject(JSONObject.toJSONString(entityMap), clazz);
                        list.add(t);
                    }
                    if (list.size() > 0) {
                        return new AggregatedPageImpl<T>(list);
                    }
                    return null;
                }
            });
            //如果不设置高亮
        } else{
            page = esTemplate.queryForPage(searchQuery, clazz);
        }
        return page.getContent();
    }

    /**
     * 根据条件查询分页列表
     * @param filter 过滤条件Map
     * @param highFields 高亮字段
     * @param sortField 排序字段
     * @param order 正序倒序
     * @param pageIndex 当前页
     * @param pageSize 分页大小
     * @return
     */
    public Map<String, Object> queryPage(Map<String,Object> filter, final List<String> highFields, String sortField, SortOrder order, int pageIndex, int pageSize) {
        NativeSearchQueryBuilder searchBuilder = new NativeSearchQueryBuilder();

        //-----------------------------高亮-----------------------------
        List<Field> fieldList = new ArrayList<>();
        if(highFields != null) {
            for (String highField : highFields) {
                fieldList.add(new HighlightBuilder.Field(highField).preTags("<em>").postTags("</em>").fragmentSize(250));
            }
        }
        searchBuilder.withHighlightFields(fieldList.toArray(new Field[fieldList.size()]));

        //-----------------------------排序-----------------------------
        if (sortField != null && order != null){
            searchBuilder.withSort(new FieldSortBuilder(sortField + ".keyword").order(order));
        }

        //-----------------------------分页-----------------------------
        searchBuilder.withPageable(PageRequest.of(pageIndex, pageSize));

        //-----------------------------过滤条件-----------------------------
        BoolQueryBuilder queryBuilder= QueryBuilders.boolQuery();
        for (Map.Entry<String, Object> entry : filter.entrySet()) {
            queryBuilder.must(QueryBuilders.matchQuery(entry.getKey(), entry.getValue()));
        }
        searchBuilder.withQuery(queryBuilder);

        //-----------------------------查询建立-----------------------------
        SearchQuery searchQuery = searchBuilder.build();
        Page<T> page = null;
        //如果设置高亮
        if (highFields != null && highFields.size() > 0) {
            page = esTemplate.queryForPage(searchQuery, clazz, new SearchResultMapper() {
                @Override
                public <T> AggregatedPage<T> mapResults(SearchResponse response,Class<T> clazz, Pageable pageable) {
                    List<T> list = new ArrayList<T>();
                    for (SearchHit searchHit : response.getHits()) {
                        if (response.getHits().getHits().length <= 0) {
                            return null;
                        }
                        Map<String, Object> entityMap = searchHit.getSourceAsMap();
                        for (String highName : highFields) {
                            String highValue = searchHit.getHighlightFields().get(highName).fragments()[0].toString();
                            entityMap.put(highName, highValue);
                        }
                        T t = JSONObject.parseObject(JSONObject.toJSONString(entityMap), clazz);
                        list.add(t);
                    }
                    if (list.size() > 0) {
                        return new AggregatedPageImpl<T>(list);
                    }
                    return null;
                }
            });
            //如果不设置高亮
        } else{
            page = esTemplate.queryForPage(searchQuery, clazz);
        }

        //-----------------------------查询数据总条数-----------------------------
        searchQuery = new NativeSearchQueryBuilder().withQuery(queryBuilder).build();
        long totalCount = esTemplate.count(searchQuery, clazz);

        //组装结果
        Map<String, Object> resultMap = new HashMap<>();
        resultMap.put("pageIndex", pageIndex);
        resultMap.put("pageSize", pageSize);
        resultMap.put("data", page.getContent());
        resultMap.put("totalCount", totalCount);
        return resultMap;
    }

}

参考文档：https://blog.csdn.net/chen_2890/article/details/83757022

　　　　　https://blog.csdn.net/lihuanlin93/article/details/83448967

猜你喜欢

转载自www.cnblogs.com/vettel0329/p/11096687.html
今日推荐
火速冲上 GitHub 热榜 —— 开源编程语言、框架哪有这么可爱？
北京人形机器人创新中心发布全球首个纯电驱拟人奔跑的全尺寸人形机器人“天工”
LFOSSA 源来如此公开课 | 掌握云原生未来：CNCF 认证全面攻略与备考秘籍
国产云输入法——仅华为无云端数据上传安全问题
开源日报 | 工业开源项目OGG 1.0；姐姐，你要和我一起配置火狐吗；苹果AI遥遥落后？Fedora 40
开放签电子签章：停止新增，优化体验，前进更进（五一假期前工作）
开源日报 | 中学生开源前端动画引擎；全球首个Llama3 8B中文版开源模型；联想电脑恐出局；Linus讽刺AI炒作
周排行
浏览器对同一域名进行请求的最大并发连接数
React Hook之自定义Hook
【转】MyBatis缓存机制
-Java-泛型
自动化测试常用脚本-发送邮件
LeetCode#859: Buddy Strings
java、Python处理字符串
第二篇の博客
Hadoop伪分布式环境安装
SQL Server进阶（十一）临时表、表变量
每日归档
更多
2024-04-27(56)
2024-04-26(39)
2024-04-25(22)
2024-04-24(36)
2024-04-23(26)
2024-04-22(39)
2024-04-21(0)
2024-04-20(6)
2024-04-19(5)
2024-04-18(0)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022