1 ES cluster structures
Description of cluster
We plan to cluster name: leyou-elastic, elasticsearch deploy three nodes, namely:
node-01: http port 9201, TCP port 9301
node-02: http port 9202, TCP port 9302
node-03: http port 9203, TCP port 9303
The first step: directly copy the day before yesterday ready ES, but before must take before copying the data clean-up
Cleanup way is to delete the data folder
Step two: Copy the folder renamed after
The third step: modify the configuration file elasticsearch.yml
Says:
http.cors.enabled: true
http.cors.allow-origin: "*"
network.host: 127.0.0.1
The name of the cluster #
cluster.name: leyou-elastic
# Current node name of each node is not the same
node.name: node-01
# Data storage path of each node is not the same
path.data: d:\class96\elasticsearch-9201\data
# Log storage path of each node is not the same
path.logs: d:\class96\elasticsearch-9201\log
External ports per node # http protocol is not the same
http.port: 9201
External ports per node # TCP protocol is not the same
transport.tcp.port: 9301
# Three nodes find each other
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9301","127.0.0.1:9302","127.0.0.1:9303"]
# Declared more than a few votes master node is valid, set (nodes / 2) + 1
discovery.zen.minimum_master_nodes: 2
Whether the master node #
node.master: true
Use utf-8 After editing a way to save it, or do not recognize Chinese
Step four: two copies again, a total of three, according to the above modification modify configuration files
Step five: three were starting ES
Step Six: Modify kibana pointing ES cluster, then start
Here pointing 920,192,029,303 is no difference
Step 7: Use elasticsearch-head plug can look at the situation cluster
2 using the operation kibana
The number of copies and the number of fragments specified index database, the default slice 5, number of copies is 1
put heima
{
"settings":{
"number_of_shards":3,
"number_of_replicas":1
}
}
Use head plug View
Native API
RestAPI
SpringDataElasticSearch way
3 RestAPI operating ES
1.1 kibana create an index database
PUT /item
{
"settings":{
"number_of_shards":3,
"number_of_replicas":1
},
"mappings": {
"docs": {
"properties": {
"id": {
"type": "keyword"
},
"title": {
"type": "text",
"analyzer": "ik_max_word"
},
"category": {
"type": "keyword"
},
"brand": {
"type": "keyword"
},
"images": {
"type": "keyword",
"index": false
},
"price": {
"type": "double"
}
}
}
}
}
1.2 created maven project
Step 1: Create maven project
Step Two: Import dependence
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.3.RELEASE</version>
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.5</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.8.1</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>6.4.3</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
1.3 Operation Code
1.3.1 Initialization client
private RestHighLevelClient client = null;
private Gson gson = new Gson();
@Before
public void init(){
client = new RestHighLevelClient(
RestClient.builder(
new HttpHost("localhost", 9201, "http"),
new HttpHost("localhost", 9202, "http"),
new HttpHost("localhost", 9203, "http")));
}
1.3.2 Adding document data
Prepare a class pojo
@Data
@AllArgsConstructor whole argument constructor // method
@NoArgsConstructor // no-argument constructor
public class Item the implements Serializable {
private Long the above mentioned id ;
private String title ; // title
private String category ; // Categories
private String Brand ; // brand
private Double . price ; // price
Private String ImagesRF Royalty Free ; // Photo address
}
// add or modify IndexRequest
Item Item = new new Item (1L, " Rice 6X mobile phone" , " mobile phone" , " millet" , 1199.0, "http.jpg" );
String jsonStr = GSON .toJson (Item);
IndexRequest Request = new new IndexRequest ( "Item" , "docs" , the item.getId () toString ().);
request.source (jsonStr, XContentType. the JSON );
Client .index (. Request, RequestOptions the DEFAULT );
1.3.3 modify the document data
What's new is the use of the above methods, it is both new modification is
1.3.4 obtain the document data according to id
GetRequest request = new GetRequest("item","docs","1");
GetResponse getResponse = client.get(request, RequestOptions.DEFAULT);
String sourceAsString = getResponse.getSourceAsString();
Item item = gson.fromJson(sourceAsString, Item.class);
System.out.println(item);
1.3.5 delete document data
DeleteRequest deleteRequest = new DeleteRequest("item","docs","1");
client.delete(deleteRequest,RequestOptions.DEFAULT);
1.3.6 New Batch document data
// prepare document data:
List <Item> List = new new ArrayList <> ();
list.add ( new new Item (1L, " millet phone 7" , " mobile phone" , " millet" , 3299.00, "HTTP: // Image .leyou.com / 13123.jpg " ));
List.add ( new new Item (2L, " nuts Rl phone " , " mobile phone " , " hammer " , 3699.00, " http://image.leyou.com/13123. JPG " ));
list.add ( new new Item (3L, " Huawei META10 " , " mobile phone ", " Huawei", 4499.00,"http://image.leyou.com/13123.jpg"));
list.add(new Item(4L, "小米Mix2S", "手机", "小米", 4299.00,"http://image.leyou.com/13123.jpg"));
list.add(new Item(5L, "荣耀V10", "手机", "华为", 2799.00,"http://image.leyou.com/13123.jpg"));
BulkRequest bulkRequest = new BulkRequest();
for (Item item : list) {
bulkRequest.add(new IndexRequest("item","docs",item.getId().toString()).source(JSON.toJSONString(item),XContentType.JSON)) ;
}
client.bulk(bulkRequest,RequestOptions.DEFAULT);
1.3.7 various queries
@Test
public void testQuery() throws Exception{
SearchRequest searchRequest = new SearchRequest("item");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(QueryBuilders.matchAllQuery());
searchSourceBuilder.query(QueryBuilders.termQuery("title","小米"));
searchSourceBuilder.query(QueryBuilders.matchQuery("title","小米手机"));
searchSourceBuilder.query(QueryBuilders.fuzzyQuery("title","大米").fuzziness(Fuzziness.ONE));
searchSourceBuilder.query(QueryBuilders.rangeQuery("price").gte(3000).lte(4000));
searchSourceBuilder.query(QueryBuilders.boolQuery().must(QueryBuilders.termQuery("title","手机"))
.must(QueryBuilders.rangeQuery("price").gte(3000).lte(3500)));
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
SearchHits searchHits = searchResponse.getHits();
long total = searchHits.getTotalHits();
System.out.println("总记录数："+total);
SearchHit[] hits = searchHits.getHits();
for (SearchHit hit : hits) {
String sourceAsString = hit.getSourceAsString();
Item item = JSON.parseObject(sourceAsString, Item.class);
System.out.println(item);
}
}
1.3.8 Filter
1, the display filter attribute field
searchSourceBuilder.fetchSource(new String[]{"title","category"},null);
searchSourceBuilder.query(QueryBuilders.matchAllQuery());
2, filter query results
searchSourceBuilder.query(QueryBuilders.termQuery("title","手机"));
searchSourceBuilder.postFilter(QueryBuilders.termQuery("brand","小米"));
1.3.9 Paging
searchSourceBuilder.query (QueryBuilders. matchAllQuery ());
searchSourceBuilder.from (0); // start position
searchSourceBuilder.size (3); // number of per page
1.3.10 Sorting
searchSourceBuilder.sort ( "ID" , the SortOrder. the ASC ); // Parameter 1: Sort 2 domain parameters: sequence
1.3.11 highlights
Construction of the conditions highlighted
searchSourceBuilder.query(QueryBuilders.termQuery("title","小米"));
HighlightBuilder highlightBuilder = new HighlightBuilder();
highlightBuilder.preTags("<font style='color:red'>");
highlightBuilder.postTags("</font>");
highlightBuilder.field("title");
searchSourceBuilder.highlighter(highlightBuilder);
Analytical results highlighted
for (SearchHit hit : hits) {
Map<String, HighlightField> highlightFields = hit.getHighlightFields();
HighlightField highlightField = highlightFields.get("title");
String title = highlightField.getFragments()[0].toString();
String sourceAsString = hit.getSourceAsString();
Item item = JSON.parseObject(sourceAsString, Item.class);
item.setTitle(title);
System.out.println(item);
}
1.3.12 polymerization
Requirements: According to statistics the number of brands
Construction of the condition code
searchSourceBuilder.query(QueryBuilders.matchAllQuery());
searchSourceBuilder.aggregation(AggregationBuilders.terms("brandAvg").field("brand"));
Analytical results:
Aggregations aggregations = searchResponse.getAggregations();
Terms terms = aggregations.get("brandAvg");
List<? extends Terms.Bucket> buckets = terms.getBuckets();
for (Terms.Bucket bucket : buckets) {
System.out.println(bucket.getKeyAsString()+":"+bucket.getDocCount());
}
Using the frame 4 SpringDataElasticSearch
1.4 Preparing the environment
1, adding a dependency
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>
2, create a bootstrap class
@SpringBootApplication
public class EsApplication {
public static void main(String[] args) {
SpringApplication.run(EsApplication.class,args);
}
}
3. Add a profile application.yml
spring:
data:
elasticsearch:
cluster-name: leyou-elastic
cluster-nodes: 127.0.0.1:9301,127.0.0.1:9302,127.0.0.1:9303
4, create a test class, into a template provided by SDE
@RunWith(SpringRunner.class)
@SpringBootTest
public class SpringDataEsManager {
@Autowired
private ElasticsearchTemplate elasticsearchTemplate;
}
Kibana：http
The original api: tcp
RestAPI：http
Sde: tcp
1.5 Operation index database and mapping
Step one: Prepare a pojo, and build relationships and indexes mapping
@Data
@AllArgsConstructor
@NoArgsConstructor
@Document(indexName="leyou",type = "goods",shards = 3,replicas = 1)
public class Goods implements Serializable{
@Field(type = FieldType.Long)
private Long id;
@Field(type = FieldType.Text,analyzer = "ik_max_word",store = true)
private String title; //标题
@Field(type = FieldType.Keyword,index = true,store = true)
private String category;// 分类
@Field(type = FieldType.Keyword,index = true,store = true)
private String brand; // 品牌
@Field(type = FieldType.Double,index = true,store = true)
private Double price; // 价格
@Field(type = FieldType.Keyword,index = false,store = true)
private String images; // Picture address
}
Step 2: Create index database and mapping
@Test
public void addIndexAndMapping () {
// elasticsearchTemplate.createIndex (Goods.class); // create the index database according to the annotations pojo
elasticsearchTemplate .putMapping (Goods. Class ); // create a map according to the annotations pojo
}
1.6 Operating documentation
// add or modify
// Goods goods = new Goods (1L , " Rice 6X mobile phone", "mobile phone", "millet", 1199.0, "http.jpg");
// goodsRespository.save (Goods); // Update or Save
// The ID query
// optional The <Goods> = optional goodsRespository.findById (1L);
// Goods Goods optional.get = ();
// System.out.println (Goods);
// delete
// goodsRespository.deleteById (1L);
// add bulk
/ * List <Goods> List = new new ArrayList <> ();
list.add (new new Goods (1L, "millet phone 7", "mobile phone", "millet" 3299.00, "http://image.leyou.com/13123.jpg"));
List.add (new new Goods (2L, "nuts Rl phone", "mobile phone", "hammer", 3699.00,"http://image.leyou.com/13123.jpg"));
list.add (new Goods (3L, "Huawei META10", "mobile phone", "Huawei", 4499.00, "http://image.leyou.com/13123.jpg"));
list.add (new new Goods (4L "millet Mix2S", "mobile phone", "millet" 4299.00, "http://image.leyou.com/13123.jpg"));
list.add (new new Goods (5L, "glory V10", "mobile phone "," Huawei, "2799.00" http://image.leyou.com/13123.jpg "));
goodsRespository.saveAll (List); * /
1.7 Queries
1.7.1 goodsRespository own inquiry
// Iterable <Goods> goodsList = goodsRespository.findAll (); // Search
// Iterable <Goods> goodsList = goodsRespository.findAll (Sort.by (Sort.Direction.ASC, "price")); // Sort
Iterable <Goods> goodsList = goodsRespository .findall (PageRequest.of (0,3)); // tab page is page number from 0. 5 representing a first page size
for (Goods Goods: goodsList) {
. the System OUT .println (Goods) ;
}
1.7.2 Custom query methods
Can define a predetermined method can be used directly in the interface
public interface GoodsRespository extends ElasticsearchRepository<Goods,Long>{
public List<Goods> findByTitle(String title);
public List<Goods> findByBrand(String brand);
public List<Goods> findByTitleOrBrand(String title,String brand);
public List<Goods> findByPriceBetween(Double low,Double high);
public List<Goods> findByBrandAndCategoryAndPriceBetween(String title,String categoty,Double low,Double high);
}
use:
// List<Goods> goodsList = goodsRespository.findByTitle("手机");
List<Goods> goodsList = goodsRespository.findByBrandAndCategoryAndPriceBetween("小米","手机",4000.0,5000.0);
for (Goods goods : goodsList) {
System.out.println(goods);
}
1.8 SpringDataElasticSearch combination of native api query
1, combined with native queries
@Test
public void testQuery(){
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
nativeSearchQueryBuilder.withQuery(QueryBuilders.termQuery("title", "小米"));
// nativeSearchQueryBuilder.withQuery(QueryBuilders.matchAllQuery());
// nativeSearchQueryBuilder.withPageable(PageRequest.of(0,3,Sort.by(Sort.Direction.DESC,"price")));
nativeSearchQueryBuilder.addAggregation(AggregationBuilders.terms("brandAvg").field("brand"));
AggregatedPage<Goods> aggregatedPage = elasticsearchTemplate.queryForPage(nativeSearchQueryBuilder.build(), Goods.class,new GoodsHighLightResultMapper());
Aggregations aggregations = aggregatedPage.getAggregations();
Terms terms = aggregations.get("brandAvg");
List<? extends Terms.Bucket> buckets = terms.getBuckets();
for (Terms.Bucket bucket : buckets) {
System.out.println(bucket.getKeyAsString()+bucket.getDocCount());
}
List<Goods> content = aggregatedPage.getContent();
for (Goods goods : content) {
System.out.println(goods);
}
}
2, deal with their own highlight
It requires a custom class that implements the processing for highlighting
class GoodsHighLightResultMapper implements SearchResultMapper{
@Override
public <T> AggregatedPage<T> mapResults(SearchResponse searchResponse, Class<T> aClass, Pageable pageable) {
List<T> content = new ArrayList<>();
Aggregations aggregations = searchResponse.getAggregations();
String scrollId = searchResponse.getScrollId();
SearchHits searchHits = searchResponse.getHits();
long total = searchHits.getTotalHits();
float maxScore = searchHits.getMaxScore();
for (SearchHit searchHit : searchHits) {
String sourceAsString = searchHit.getSourceAsString();
T t = JSON.parseObject(sourceAsString, aClass);
Map<String, HighlightField> highlightFields = searchHit.getHighlightFields();
HighlightField highlightField = highlightFields.get("title");
String title = highlightField.getFragments()[0].toString();
try {
BeanUtils.setProperty(t,"title",title);
} catch (Exception e) {
e.printStackTrace();
}
content.add(t);
}
return new AggregatedPageImpl<T>(content,pageable,total,aggregations,scrollId,maxScore);
// List<T> content, Pageable pageable, long total, Aggregations aggregations, String scrollId, float maxScore
}
}
3, using
