The pit that elasticsearch stepped on when deleting the index and then creating a new one

course of events

All operations are in the elasticsearchpluginhead

1. Clear all indexes

rm -rf /This delete operation is as simple as linux

DELETE
\*

It is Jiangzi, click submit, and all will be deleted. You can refresh the page and take a look.

2. Import data

How our data got to ES

In the project, when we query ES data, it is passed spring data jpa, but the insertion is not. It is a timed task to pull the third-party platform. The amount of data is relatively large, and it is integrated kafkaand then logstashsent to ES. kafkaIt is a string in json format sent directly, logstashand a dynamic template is configured in it. It can be said that it is almost the same as the default.

Problem occurred

It can be said that each object (should be called document) sent to ES has a lot of fields, most of which do not need word segmentation, but dynamic templates will not judge, so the data in ES is word segmentation! Then we use the aggregation query part of the code like this:

TermsBuilder provinceTermsBuilder = AggregationBuilders.terms("provinceAgg").field("provinceCode.raw");
TermsBuilder cityTermsBuilder = AggregationBuilders.terms("cityAgg").field("cityCode.raw");

Well, we can't find any data.
Take a look .raw, the official explanation is roughly summed up, that is, the same field, one more way of mapping. But we have not set this mapping! We don't have provinceCode.rawAND cityCode.rawfields, so querying by these two fields doesn't have any data at all

The codes of the provinces and cities represented by these two fields, for example, 31 represents Shanghai. This does not require a participle.

3. Research Index

The cause of the problem has been found. I researched how to do this mapping. After reading the official document , I remembered that a mapped json format file was given during the outsourcing handover. Check it out. It should be the mapping of these fields, but it may be a problem of version or document update. After executing it directly , did not work, according to the official document in the document to make the next modification, OK probably the process is

1. delete all indexes again
2. Create new index and set up mapping

When solving the index problem, there is also the problem that the index cannot be found when querying the data. It is found spring data jpathat the ES index name configured in the discovery is different from the actual index. Google it, and sure enough, there is also the concept of aliases in ES. Quoting the official website:

Even if you think that the index design is perfect now, in a production environment, there may still be some changes that need to be made. Be prepared: use aliases instead of index names in your app. Then you can rebuild the index at any time. Aliases have little overhead and should be used widely

4. There are also operations such as initial values!

Still no data!
A new feature was added at handover, so a new field was added. But this field has a default value! So there is no data when querying this field.
Default value settings:

METHOD: POST
URL: /{索引}/{字段}/_update_by_query
{"script":{"inline":"ctx._source.{字段}=0"}}

Be careful to use postthe method. If you accidentally use it, you will putdirectly create a new {field} type. Then there are various problems when querying this field. Reluctantly delete it!

Summarize

When using ESdevelopment, you need to pay attention to the backup configuration:

1. index mapping
2. index alias

Pay attention to the http method when debugging development. If you accidentally use DELETEit, delete it.

The original text was first published on Fengbei 's blog