之前研究了一下es JAVA API客户端，最近需要做成web项目，使用SpringBoot来整合ES，经过一段时间的研究记录一下学习过程。
下面首先给出一些参考链接：
ES JAVA API https://www.elastic.co/guide/en/elasticsearch/client/java-api/6.3/index.html
SpringBoot+ES: https://www.cnblogs.com/guozp/archive/2018/04/02/8686904.html
SpringBoot ES官方文档： https://docs.spring.io/spring-data/elasticsearch/docs/3.1.5.RELEASE/reference/html/
这里SpringBoot使用2.1.3.RELEASE版本，Spring Data Elasticsearch使用Version 3.1.5.RELEASE，ES使用6.3.2版本。
maven依赖配置
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.3.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.example</groupId>
<artifactId>vehicle</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>vehicle</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<!--定时任务和@Slf4j注解日志的依赖-->
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
导入SpringBoot需要的包以及spring-boot-starter-data-elasticsearch，对于包的引入一定要谨慎，避免包冲突。
参数配置
##\u7AEF\u53E3\u53F7
server.port=8888
# ES
#\u5F00\u542F Elasticsearch \u4ED3\u5E93(\u9ED8\u8BA4\u503C:true)
spring.data.elasticsearch.repositories.enabled=true
#\u9ED8\u8BA4 9300 \u662F Java \u5BA2\u6237\u7AEF\u7684\u7AEF\u53E3\u30029200 \u662F\u652F\u6301 Restful HTTP \u7684\u63A5\u53E3
spring.data.elasticsearch.cluster-nodes = 10.3.10.144:9300
spring.data.elasticsearch.cluster-name = es
#spring.data.elasticsearch.cluster-nodes \u96C6\u7FA4\u8282\u70B9\u5730\u5740\u5217\u8868\uFF0C\u7528\u9017\u53F7\u5206\u9694\u3002\u5982\u679C\u6CA1\u6709\u6307\u5B9A\uFF0C\u5C31\u542F\u52A8\u4E00\u4E2A\u5BA2\u6237\u7AEF\u8282\u70B9
#spring.data.elasticsearch.propertie \u7528\u6765\u914D\u7F6E\u5BA2\u6237\u7AEF\u7684\u989D\u5916\u5C5E\u6027
#\u5B58\u50A8\u7D22\u5F15\u7684\u4F4D\u7F6E
#spring.data.elasticsearch.properties.path.home=/data/project/target/elastic
#\u8FDE\u63A5\u8D85\u65F6\u7684\u65F6\u95F4
#spring.data.elasticsearch.properties.transport.tcp.connect_timeout=120s核心参数：主要配置ES连接信息
spring.data.elasticsearch.repositories.enabled
spring.data.elasticsearch.cluster-nodes
spring.data.elasticsearch.cluster-name
Model层
这里设计了一个VehicleList类来进行测试，首先在ES中创建Index,Type等元数据Mapping。
这里Mapping使用ES JAVA API来实现，spring-boot-starter-data-elasticsearch应该也可以实现，以后研究一下。
@Before
public void init() throws Exception {
//设置集群名称
Settings settings = Settings.builder().
put("cluster.name", "es")
//.put("client.transport.sniff", true) //自动感知的功能（可以通过当前指定的节点获取所有es节点的信息）
.build();
//创建client
client = new PreBuiltTransportClient(settings).addTransportAddresses(
new TransportAddress(InetAddress.getByName("10.3.10.144"), 9300));
}
@Test
public void testSettingVehicleListMappings() throws IOException {
//1:settings
HashMap<String, Object> settings_map = new HashMap<String, Object>(2);
settings_map.put("number_of_shards", 3);
settings_map.put("number_of_replicas", 1);
//2:mappings
XContentBuilder builder = XContentFactory.jsonBuilder()
//开始设置
.startObject()//
.field("dynamic", "true")
//只设置字段 不传参
.startObject("properties")
.startObject("id")
.field("type", "text")
.field("index", "true")
//.field("store", "yes")
.endObject()
.startObject("if_online") // 0在线 1 不在线
.field("type", "integer")
.endObject()
.startObject("online_time")
.field("type", "long")
.endObject()
.startObject("if_active") //0激活 1不激活
.field("type", "integer")
.endObject()
.startObject("active_time")
.field("type", "long")
.endObject()
.endObject()
.endObject();
CreateIndexRequestBuilder prepareCreate = client.admin().indices().prepareCreate("vehicle_list");
prepareCreate.setSettings(settings_map).addMapping("vehicle", builder).get();
}创建Model与Mapping相对应 注意Document注解使用
@Document(indexName = "vehicle_list", type = "vehicle")
public class VehicleList implements Serializable {
private String id;
private int ifOnline;
private long onlineTime;
private int ifActive;
private long activeTime;
public VehicleList(){}
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public int getIfOnline() {
return ifOnline;
}
public void setIfOnline(int ifOnline) {
this.ifOnline = ifOnline;
}
public long getOnlineTime() {
return onlineTime;
}
public void setOnlineTime(long onlineTime) {
this.onlineTime = onlineTime;
}
public int getIfActive() {
return ifActive;
}
public void setIfActive(int ifActive) {
this.ifActive = ifActive;
}
public long getActiveTime() {
return activeTime;
}
public void setActiveTime(long activeTime) {
this.activeTime = activeTime;
}
}Dao层设计
@Component
public interface VehicleListRepository extends ElasticsearchRepository<VehicleList, String> {
public long countByIfActive(Integer ifActive);
public long countByIfOnline(Integer ifOnline);
}
Controller层设计
新增方法
@GetMapping("saveVehicleList")
public String saveVehicleList(){
for(int i=0; i<100; i++){
VehicleList v = new VehicleList();
v.setId(produceVehicleId());
v.setIfActive(r.nextInt(2));
v.setIfOnline(r.nextInt(2));
v.setOnlineTime(System.currentTimeMillis());
v.setActiveTime(System.currentTimeMillis());
vehicleListRepository.save(v);
}
return "saveVehicleList success";
}分页查询实现
//http://localhost:8888/getVehicleList?pageNumber=0&pageSize=20
@GetMapping("getVehicleList")
public List<VehicleList> getVehicleList(Integer pageNumber, Integer pageSize){
Page<VehicleList> vehicleListPage = vehicleListRepository.search(
QueryBuilders.matchAllQuery(), new PageRequest(pageNumber, pageSize));
//log.info("totalNum = " + vehicleListPage.getTotalElements());
return vehicleListPage.getContent();
}
//http://localhost:8888/getVehicleInfoList?pageNumber=0&pageSize=20&vehicleId='京A25906'
@GetMapping("getVehicleInfoList")
public List<VehicleInfo> getVehicleInfoList(String vehicleId, Integer pageNumber, Integer pageSize){
Page<VehicleInfo> vehicleInfoPage = vehicleInfoRepository.search(
QueryBuilders.commonTermsQuery("vehicleId", vehicleId), new PageRequest(pageNumber, pageSize));
//log.info("totalNum = " + vehicleInfoPage.getTotalElements());
return vehicleInfoPage.getContent();
}以上就是基本的代码，代码较简单，剩余就是主要研究一下Dao层Spring ES对于数据操作提供的操作接口。最近读了一部分源码，有一些感悟，感觉当学习一种新技术时，刚开始可以参考一下博文，后面最好还是参考官方文档以及源码，不同版本的变化有点大，坑太多。
代码已经上传GitHub: https://github.com/ChenWenKaiVN/SpringBootElacticSearch
下面研究一下spring-boot-starter-data-elasticsearch源码：
核心包spring-data-elasticsearch-3.1.5.RELEASE.jar
客户端构建TransportClient
构造client对于配置文件参数的解析：org.springframework.data.elasticsearch.config TransportClientBeanDefinitionParser
由此可以得知配置文件中的可选参数
private void setConfigurations(Element element, BeanDefinitionBuilder builder) {
builder.addPropertyValue("clusterNodes", element.getAttribute("cluster-nodes"));
builder.addPropertyValue("clusterName", element.getAttribute("cluster-name"));
builder.addPropertyValue("clientTransportSniff", Boolean.valueOf(element.getAttribute("client-transport-sniff")));
builder.addPropertyValue("clientIgnoreClusterName", Boolean.valueOf(element.getAttribute("client-transport-ignore-cluster-name")));
builder.addPropertyValue("clientPingTimeout", element.getAttribute("client-transport-ping-timeout"));
builder.addPropertyValue("clientNodesSamplerInterval", element.getAttribute("client-transport-nodes-sampler-interval"));
}客户端的构建： org.springframework.data.elasticsearch.client TransportClientFactoryBean
protected void buildClient() throws Exception {
client = new PreBuiltTransportClient(settings());
clusterNodes.stream() //
.peek(it -> logger.info("Adding transport node : " + it.toString())) //
.forEach(client::addTransportAddress);
client.connectedNodes();
}
private Settings settings() {
if (properties != null) {
Settings.Builder builder = Settings.builder();
properties.forEach((key, value) -> {
builder.put(key.toString(), value.toString());
});
return builder.build();
}
return Settings.builder()
.put("cluster.name", clusterName)
.put("client.transport.sniff", clientTransportSniff)
.put("client.transport.ignore_cluster_name", clientIgnoreClusterName)
.put("client.transport.ping_timeout", clientPingTimeout)
.put("client.transport.nodes_sampler_interval", clientNodesSamplerInterval)
.build();
}可知没有参数配置则加载默认配置
注解
org.springframework.data.elasticsearch.annotations
Document注解，用来映射Model与Es中Mapping
@Persistent
@Inherited
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE})
public @interface Document {
String indexName();
String type() default "";
boolean useServerConfiguration() default false;
short shards() default 5;
short replicas() default 1;
String refreshInterval() default "1s";
String indexStoreType() default "fs";
boolean createIndex() default true;
}Field注解用来映射属性
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
@Documented
@Inherited
public @interface Field {
FieldType type() default FieldType.Auto;
boolean index() default true;
DateFormat format() default DateFormat.none;
String pattern() default "";
boolean store() default false;
boolean fielddata() default false;
String searchAnalyzer() default "";
String analyzer() default "";
String normalizer() default "";
String[] ignoreFields() default {};
boolean includeInParent() default false;
String[] copyTo() default {};
}Dao层接口
VehicleListRepository继承ElasticsearchRepository
@Component
public interface VehicleListRepository extends ElasticsearchRepository<VehicleList, String> {
public long countByIfActive(Integer ifActive);
public long countByIfOnline(Integer ifOnline);
}ElasticsearchRepository继承ElasticsearchCrudRepository
@NoRepositoryBean
public interface ElasticsearchRepository<T, ID extends Serializable> extends ElasticsearchCrudRepository<T, ID> {
<S extends T> S index(S entity);
Iterable<T> search(QueryBuilder query);
Page<T> search(QueryBuilder query, Pageable pageable);
Page<T> search(SearchQuery searchQuery);
Page<T> searchSimilar(T entity, String[] fields, Pageable pageable);
void refresh();
Class<T> getEntityClass();
}ElasticsearchCrudRepository继承PagingAndSortingRepository
@NoRepositoryBean
public interface ElasticsearchCrudRepository<T, ID extends Serializable> extends PagingAndSortingRepository<T, ID> {
}PagingAndSortingRepository继承CrudRepository
@NoRepositoryBean
public interface PagingAndSortingRepository<T, ID> extends CrudRepository<T, ID> {
/**
* Returns all entities sorted by the given options.
*
* @param sort
* @return all entities sorted by the given options
*/
Iterable<T> findAll(Sort sort);
/**
* Returns a {@link Page} of entities meeting the paging restriction provided in the {@code Pageable} object.
*
* @param pageable
* @return a page of entities
*/
Page<T> findAll(Pageable pageable);
}CrudRepository继承Repository
@NoRepositoryBean
public interface CrudRepository<T, ID> extends Repository<T, ID> {
/**
* Saves a given entity. Use the returned instance for further operations as the save operation might have changed the
* entity instance completely.
*
* @param entity must not be {@literal null}.
* @return the saved entity will never be {@literal null}.
*/
<S extends T> S save(S entity);
/**
* Saves all given entities.
*
* @param entities must not be {@literal null}.
* @return the saved entities will never be {@literal null}.
* @throws IllegalArgumentException in case the given entity is {@literal null}.
*/
<S extends T> Iterable<S> saveAll(Iterable<S> entities);
/**
* Retrieves an entity by its id.
*
* @param id must not be {@literal null}.
* @return the entity with the given id or {@literal Optional#empty()} if none found
* @throws IllegalArgumentException if {@code id} is {@literal null}.
*/
Optional<T> findById(ID id);
/**
* Returns whether an entity with the given id exists.
*
* @param id must not be {@literal null}.
* @return {@literal true} if an entity with the given id exists, {@literal false} otherwise.
* @throws IllegalArgumentException if {@code id} is {@literal null}.
*/
boolean existsById(ID id);
/**
* Returns all instances of the type.
*
* @return all entities
*/
Iterable<T> findAll();
/**
* Returns all instances of the type with the given IDs.
*
* @param ids
* @return
*/
Iterable<T> findAllById(Iterable<ID> ids);
/**
* Returns the number of entities available.
*
* @return the number of entities
*/
long count();
/**
* Deletes the entity with the given id.
*
* @param id must not be {@literal null}.
* @throws IllegalArgumentException in case the given {@code id} is {@literal null}
*/
void deleteById(ID id);
/**
* Deletes a given entity.
*
* @param entity
* @throws IllegalArgumentException in case the given entity is {@literal null}.
*/
void delete(T entity);
/**
* Deletes the given entities.
*
* @param entities
* @throws IllegalArgumentException in case the given {@link Iterable} is {@literal null}.
*/
void deleteAll(Iterable<? extends T> entities);
/**
* Deletes all entities managed by the repository.
*/
void deleteAll();
}@Indexed
public interface Repository<T, ID> {
}以上便是Dao层的类继承结构，可以清楚了解默认支持的操作方法，另外也支持Spring JPA形式的操作接口，可以根据方法名来进行查询映射，特别方便，具体可以参考：https://www.cnblogs.com/guozp/archive/2018/04/02/8686904.html
方法和es查询转换：
| Keyword | Sample | Elasticsearch Query String |
|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
直接使用代码来查询：
interface PersonRepository extends Repository<User, Long> {
List<Person> findByEmailAddressAndLastname(EmailAddress emailAddress, String lastname);
// Enables the distinct flag for the query
List<Person> findDistinctPeopleByLastnameOrFirstname(String lastname, String firstname);
List<Person> findPeopleDistinctByLastnameOrFirstname(String lastname, String firstname);
// Enabling ignoring case for an individual property
List<Person> findByLastnameIgnoreCase(String lastname);
// Enabling ignoring case for all suitable properties
List<Person> findByLastnameAndFirstnameAllIgnoreCase(String lastname, String firstname);
// Enabling static ORDER BY for a query
List<Person> findByLastnameOrderByFirstnameAsc(String lastname);
List<Person> findByLastnameOrderByFirstnameDesc(String lastname);
}用Java8 Stream查询和sql语句查询
| 1 2 3 4 5 6 7 |
|
使用@Query注解进行查询
public interface BookRepository extends ElasticsearchRepository<Book, String> {
@Query("{"bool" : {"must" : {"field" : {"name" : "?0"}}}}")
Page<Book> findByName(String name,Pageable pageable);
}
具体的高级用法,可以通过封装QueryBuilder以及SearchQuery来实现
public interface ElasticsearchRepository<T, ID extends Serializable> extends ElasticsearchCrudRepository<T, ID> {
<S extends T> S index(S entity);
Iterable<T> search(QueryBuilder query);
Page<T> search(QueryBuilder query, Pageable pageable);
Page<T> search(SearchQuery searchQuery);
Page<T> searchSimilar(T entity, String[] fields, Pageable pageable);
void refresh();
Class<T> getEntityClass();
}不过官方目前好像是不鼓励使用TranportClient来查询，支持使用restful api来操作。