Author: Robod
link: https: //zhuanlan.zhihu.com/p/261054951
Source: know almost
copyrighted by the author. For commercial reprints, please contact the author for authorization, and for non-commercial reprints, please indicate the source.
Brand Statistics
When we search for smartphones on JD.com, relevant brands will be listed for users to choose
What we want to achieve is also this function, which is to classify and count the brands in the search results
This function is the same as the classification statistics mentioned in the previous article, so add a few lines of code to get it done.
But when I searched for "smartphone", only two brands appeared, which obviously did not match the actual situation. The reason is simple, "ik_smart" does not split "smartphone" into "smart" and "mobile", so just change the word segmentation mode to "ik_max_word". Change the name field in SkuInfo from "ik_smart" to "ik_max_word", then re-import the data, and test it again:
That's right.
Specification statistics
When we search on JD.com, the specification information will be listed for users to choose. There is also specification information in our ES, but these information are all json strings. What we have to do is to convert these json strings into a Map collection to achieve the same function as JD.
Add the following code to the searchByKeywords method of SkuEsServiceImpl:
As can be seen from the code, first add search conditions, then take out the set of spec from the search results, traverse and store it into specMap. Because the search result is a json string, each time the json string is converted into a map set, and then the map is traversed, and the data is taken out from the map and put into the specSet in turn. If there is no corresponding specSet in the specMap, it will be directly new and stored in it, and if it is, it will be directly taken out of the specMap. Finally, put the specMap into the SearchEntity and return it to the front end.
Condition screening
Classification and brand filtering
When we use brand or category as a conditional search, we don’t have to deal with brand and category statistics. Before there was only one keywords parameter, I wrote it directly in the address bar, but now that there are more parameters, it is better to encapsulate it into SearchEntity, and then write the parameters in the request body.
private String keywords; //前端传过来的关键词信息
private String brand; //前端传过来的品牌信息
private String category; //前端传过来的分类信息
Then add code to the searchByKeywords method of SkuEsServiceImpl . My current searchByKeywords has been written very bloated. Let me ignore this problem, and finally optimize the code.
If the category or brand parameter is not empty, it will be used as a conditional filter and no statistics will be performed, otherwise statistics will be performed. It should be noted that the withFilter() method is used here. In fact, withQuery() is also possible, but with withQuery(), the highlight search will not work, so use it withFilter().
It can be seen that now that the brand information is specified but the classification information is not specified, the brand will not be counted, and the classification will still be counted, which has achieved our expected effect.
Specification filter
Same as the previous one, when we pass the specifications as a parameter to the backend, we also won't perform specification statistics. To achieve this function, you must first add a field in SearchEntity to receive specification parameters.
private List<String> searchSpec; //前端传过来的规格信息
Then add code in the searchByKeywords method of SkuEsServiceImpl :
……
Map<String,String> searchSpec = searchEntity.getSearchSpec();
if (searchSpec != null && searchSpec.size() > 0) {
for (String key:searchSpec.keySet()){
//格式：specMap.规格名字.keyword keyword表示不分词
boolQueryBuilder.filter(QueryBuilders.termQuery("specMap."+key+".keyword",searchSpec.get(key)));
}
}
……
Get the searchSpec passed from the front end, then traverse the content of the specification, and then use boolQueryBuilder.filter() to filter.
It can be seen from the figure that when we specify the color as blue and the version as "6GB+64GB", the results are all the results we have filtered.
Price filter
When we search for products on JD.com, we can specify a price range. The same function is to be realized now. First of all, we still have to add a field in SearchEntity to receive interval parameters.
private String price; //前端穿过来的价格区间字符串 300-500元 3000元以上
Then add the implemented code:
……
if (!StringUtils.isEmpty(searchEntity.getPrice())) {
String[] price = searchEntity.getPrice().replace("元","")
.replace("以上","").split("-");
boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").gte(Integer.parseInt(price[0])));
if (price.length>1){
boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").lt(Integer.parseInt(price[1])));
}
}
……
In this way, the price filtering can be achieved.
Paging function
The paging function is relatively simple, receiving the pageNum parameter passed in from the front end, and then calling the nativeSearchQueryBuilder.withPageable() method to achieve paging.
int pageNum = 1;
if (!StringUtils.isEmpty(searchEntity.getPageNum())) {
pageNum = Integer.parseInt(searchEntity.getPageNum());
}
nativeSearchQueryBuilder.withPageable(PageRequest.of(pageNum-1,SEARCH_PAGE_SIZE));
The code is very simple. Give pageNum a default value. If the front end does not pass a parameter, the first page will be displayed, and the number of entries per page is 10. I defined a constant SEARCH_PAGE_SIZE to represent 10.
It can be seen that a total of 22 pieces of data are searched, and 10 pieces per page are 3 pages, and the paging is correct.
Sort
The sorting function is relatively simple, that is, pass the field to be sorted and the sorting method to the corresponding method.
String sortField = searchEntity.getSortField();
String sortRule = searchEntity.getSortRule();
if (!StringUtils.isEmpty(sortField) && !StringUtils.isEmpty(sortRule)) {
nativeSearchQueryBuilder.withSort(SortBuilders.fieldSort(sortField).order(SortOrder.valueOf(sortRule)));
}
sortField is the field to be sorted, and sortRule is the way to sort. If these two parameters are not empty, just call the withSort method to add search conditions. that's it.
Highlight
When we usually search on Jingdong or Taobao, the keywords will be highlighted.
In fact, it is very simple to implement, as long as the search term is wrapped in html and the style is changed to red.
First, you need to configure the highlight , that is, specify the highlight domain:
HighlightBuilder.Field field = new HighlightBuilder.Field("name"); //指定域
field.preTags("<em style=\"color:red;\">"); //指定前缀
field.postTags("</em>"); //指定后缀
nativeSearchQueryBuilder.withHighlightFields(field);
In this code, we specified the domain to be highlighted and added the highlighted html code to it.
After specifying the domain to be highlighted , you need to perform a highlight search, and replace the non-highlighted data with the highlighted data . The queryForPage(SearchQuery query, Class<T> clazz)method was used for searching before , now it is changed to queryForPage(SearchQuery query, Class<T> clazz, SearchResultMapper mapper)method.
AggregatedPage<SkuInfo> skuInfos = elasticsearchTemplate
.queryForPage(nativeSearchQuery, SkuInfo.class, new SearchResultMapper() {
@Override
public <T> AggregatedPage<T> mapResults(SearchResponse response, Class<T> clazz, Pageable pageable) {
List<T> list = new ArrayList<>();
for (SearchHit hit : response.getHits()) { //遍历所有数据
SkuInfo skuInfo = JSON.parseObject(hit.getSourceAsString(), SkuInfo.class);//非高亮数据
HighlightField highlightField = hit.getHighlightFields().get("name"); //高亮数据
//将非高亮数据替换成高亮数据
if (highlightField != null && highlightField.getFragments() != null) {
Text[] fragments = highlightField.getFragments();
StringBuilder builder = new StringBuilder();
for (Text fragment : fragments) {
builder.append(fragment.toString());
}
skuInfo.setName(builder.toString());
list.add((T) skuInfo);
}
}
return new AggregatedPageImpl<T>(list, pageable,
response.getHits().getTotalHits(),response.getAggregations());
}
});
When I ran it, I found that the filter search skuInfos.getAggregations()always reported a null pointer exception. I thought it was because the highlight search and the filter search couldn't be used together. Finally, I found that I had response.getAggregations()lost the writing here , and one parameter was missing. So pay attention here.
Code optimization
When I watched the video before, I found that he did not write very well. In fact, it only needs to check it once, but it has been checked many times. However, this problem is also mentioned at the end of the video and the code is improved. So I didn’t follow the instructions in the video before, but squeezed into one method and thought about optimizing at the end. The last method was written in more than 100 lines, which was too bloated, and in the "Alibaba Java Development Manual" It is also mentioned that a method should not exceed 80 lines. Now that the search function has been implemented, you can optimize it. Below is the code I optimized
@Override
public SearchEntity searchByKeywords(SearchEntity searchEntity) {
if (searchEntity != null && !StringUtils.isEmpty(searchEntity.getKeywords())) {
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
//配置高亮
HighlightBuilder.Field field = new HighlightBuilder.Field("name");
field.preTags("<em style=\"color:red;\">");
field.postTags("</em>");
nativeSearchQueryBuilder.withHighlightFields(field);
//条件筛选或者分组统计
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
if (!StringUtils.isEmpty(searchEntity.getCategory())) { //分类过滤
boolQueryBuilder.filter(QueryBuilders.termQuery("categoryName", searchEntity.getCategory()));
} else {
nativeSearchQueryBuilder.addAggregation(AggregationBuilders.terms
("categories_grouping").field("categoryName").size(10000));
}
if (!StringUtils.isEmpty(searchEntity.getBrand())) { //品牌过滤
boolQueryBuilder.filter(QueryBuilders.termQuery("brandName", searchEntity.getBrand()));
} else {
nativeSearchQueryBuilder.addAggregation(AggregationBuilders.terms
("brands_grouping").field("brandName").size(10000));
}
nativeSearchQueryBuilder.addAggregation(AggregationBuilders.terms
("spec_grouping").field("spec.keyword").size(10000));
if (!StringUtils.isEmpty(searchEntity.getPrice())) { //价格过滤
String[] price = searchEntity.getPrice().replace("元", "")
.replace("以上", "").split("-");
boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").gte(Integer.parseInt(price[0])));
if (price.length > 1) {
boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").lt(Integer.parseInt(price[1])));
}
}
Map<String, String> searchSpec = searchEntity.getSearchSpec();
if (searchSpec != null && searchSpec.size() > 0) {
for (String key : searchSpec.keySet()) {
boolQueryBuilder.filter(QueryBuilders.termQuery("specMap." + key + ".keyword", searchSpec.get(key)));
}
}
//分页
int pageNum = (!StringUtils.isEmpty(searchEntity.getPageNum()))
? (Integer.parseInt(searchEntity.getPageNum())) : 1;
nativeSearchQueryBuilder.withPageable(PageRequest.of(pageNum - 1, SEARCH_PAGE_SIZE));
//排序
String sortField = searchEntity.getSortField();
String sortRule = searchEntity.getSortRule();
if (!StringUtils.isEmpty(sortField) && !StringUtils.isEmpty(sortRule)) {
nativeSearchQueryBuilder.withSort(SortBuilders.fieldSort(sortField).order(SortOrder.valueOf(sortRule)));
}
nativeSearchQueryBuilder
.withQuery(QueryBuilders.queryStringQuery(searchEntity.getKeywords()).field("name"))
.withFilter(boolQueryBuilder); //这两行顺序不能颠倒
AggregatedPage<SkuInfo> skuInfos = elasticsearchTemplate
.queryForPage(nativeSearchQueryBuilder.build(), SkuInfo.class, new SearchResultMapperImpl());
Aggregations aggregations = skuInfos.getAggregations();
List<String> categoryList = buildGroupList(aggregations.get("categories_grouping"));
List<String> brandList = buildGroupList(aggregations.get("brands_grouping"));
Map<String, Set<String>> specMap = specGroup(aggregations.get("spec_grouping"));
searchEntity.setTotal(skuInfos.getTotalElements());
searchEntity.setTotalPages(skuInfos.getTotalPages());
searchEntity.setCategoryList(categoryList);
searchEntity.setBrandList(brandList);
searchEntity.setSpecMap(specMap);
searchEntity.setRows(skuInfos.getContent());
return searchEntity;
}
return null;
}
//将过滤搜索出来的StringTerms转换成List集合
private List<String> buildGroupList(StringTerms stringTerms) {
List<String> list = new ArrayList<>();
if (stringTerms != null) {
for (StringTerms.Bucket bucket : stringTerms.getBuckets()) {
list.add(bucket.getKeyAsString());
}
}
return list;
}
//规格统计
private Map<String, Set<String>> specGroup(StringTerms specTerms) {
Map<String, Set<String>> specMap = new HashMap<>(16);
for (StringTerms.Bucket bucket : specTerms.getBuckets()) {
Map<String, String> map = JSON.parseObject(bucket.getKeyAsString(), Map.class);
for (String key : map.keySet()) {
Set<String> specSet;
if (!specMap.containsKey(key)) {
specSet = new HashSet<>();
specMap.put(key, specSet);
} else {
specSet = specMap.get(key);
}
specSet.add(map.get(key));
}
}
return specMap;
}
The SearchResultMapper parameter in queryForPage was picked up separately by me:
public class SearchResultMapperImpl implements SearchResultMapper {
@Override
public <T> AggregatedPage<T> mapResults(SearchResponse response, Class<T> clazz, Pageable pageable) {
List<T> list = new ArrayList<>();
for (SearchHit hit : response.getHits()) { //遍历所有数据
SkuInfo skuInfo = JSON.parseObject(hit.getSourceAsString(), SkuInfo.class);//非高亮数据
HighlightField highlightField = hit.getHighlightFields().get("name"); //高亮数据
//将非高亮数据替换成高亮数据
if (highlightField != null && highlightField.getFragments() != null) {
Text[] fragments = highlightField.getFragments();
StringBuilder builder = new StringBuilder();
for (Text fragment : fragments) {
builder.append(fragment.toString());
}
skuInfo.setName(builder.toString());
list.add((T) skuInfo);
}
}
return new AggregatedPageImpl<T>(list, pageable,
response.getHits().getTotalHits(),response.getAggregations());
}
}
Although the code is still quite a lot, the logic is much clearer. Although it can be optimized, I don't think it is necessary. Now test it:
Well, all functions are running normally.
summary
The previous article simply set up the environment, and only did a keyword search and classified statistics function. Then this article is a supplement to the previous article. The functions of brand statistics, specification statistics, condition filtering, paging, sorting and highlighting are realized. At this point, the function of product search is all completed.