ElasticSearch (4) --- Bulk bulk operations

Previous: elasticsearch (. 3) Operation -CURL

1.Bulk format

{action:{metadata}}\n  //注意 “\n”表示换行符
{requstbody}\n (请求体)

grammar	description
action	(Behavior), (created when the document does not exist), update (update document), index (create a new document or replace with a document), delete (delete a document) that contains create.
metadata	(Specific operation behavior index information), need to specify the data _index, _type, _id.

difference and create the index: If data exists, use the create operation fails, it will prompt existing document, use the index can be executed successfully.

Example:

1. Bulk Insert

There is now a file books.jsonin batch data needs to be written, the data in the file as follows:

{"index":{"_index":"books","_type":"info","_id":"1"}}
{"name":"西游记","author":"吴承恩","price":"40"}
{"index":{"_index":"books","_type":"info","_id":"2"}}
{"name":"三国演义","author":"罗贯中","price":"41"}
{"index":{"_index":"books","_type":"info","_id":"3"}}
{"name":"水浒传","author":"施耐庵","price":"42"}
{"index":{"_index":"books","_type":"info","_id":"4"}}
{"name":"红楼梦","author":"曹雪芹","price":"43"} //注意：此处还需要点下回车键，否则会报错

Use Xftpimport this file linuxin the /home/zhangsan/data/directory:

In linuxthe following command input

curl -H 'Content-Type:application/json' -XPOST 'http://120.76.217.14:9200/_bulk?pretty' --data-binary '@/home/zhangsan/data/books.json'

And then query the index:

[zhangsan@tomcat-tst data]$ curl -XGET 'http://120.76.217.14:9200/books/info/_search?pretty'
{
  "took" : 813,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 4,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "books",
        "_type" : "info",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "name" : "西游记",
          "author" : "吴承恩",
          "price" : "40"
        }
      },
      {
        "_index" : "books",
        "_type" : "info",
        "_id" : "2",
        "_score" : 1.0,
        "_source" : {
          "name" : "三国演义",
          "author" : "罗贯中",
          "price" : "41"
        }
      },
      {
        "_index" : "books",
        "_type" : "info",
        "_id" : "3",
        "_score" : 1.0,
        "_source" : {
          "name" : "水浒传",
          "author" : "施耐庵",
          "price" : "42"
        }
      },
      {
        "_index" : "books",
        "_type" : "info",
        "_id" : "4",
        "_score" : 1.0,
        "_source" : {
          "name" : "红楼梦",
          "author" : "曹雪芹",
          "price" : "43"
        }
      }
    ]
  }
}
[zhangsan@tomcat-tst data]$

Batch write success,

2. Batch Processing

{"update":{"_index":"books","_type":"info","_id":"1"}} //更新文档
{"doc":{"name":"人性的弱点","author":"卡耐基"}} //请求体
{"delete":{"_index":"books","_type":"info","_id":"2"}} //删除不需要请求体
{"create":{"_index":"books","_type":"info","_id":"10"}} //当文档不存在的时候创建文档
{"name":"孙子兵法","author":"孙武","price":"42"} //请求体
{"index":{"_index":"books","_type":"info","_id":"3"}} //创建新文档或者替换已用文档
{"name":"厚黑学","author":"李宗吾","price":"50"} //请求体

Bulk above is substantially in a batch operation 4 action.

3. The amount of data processed batch

  Bulk operation because the import data will be processed into memory, so the amount of data processing is limited. How much can handle the amount of data depending on the hardware configuration, size and complexity of your document, index, and search loads.

  General recommendations are 1000-5000 document, the size of the proposal is 5-15MB, the default can not exceed 100M, you can configure the profile es (config elasticsearch.yml under) in.

4. Unique designated/index/type

  In more of the above batch operations on the document request body, we have the same /books/info, we can in curlthe URLspecified / books / info, so you can still cover the metadata row _index and _type, but it will use the URL the metadata values as default:

POST /books/info
{"index":{"_id":"3"}} //创建新文档或者替换已用文档
{"name":"厚黑学","author":"李宗吾","price":"50"} //请求体

Next: elasticsearch (5) Plug the -head