Elasticsearch series - Cluster API (a)

Disclaimer: This article is a blogger original article, follow the CC 4.0 BY-SA copyright agreement, reproduced, please attach the original source link and this statement.

This link: https://blog.csdn.net/projim_tao/article/details/102644646

introduction

Elasticsearch provides a number of functions related to the Rest API Elastic search operations, providing great convenience, grasp the premise of these API is proficient in the use of Elasticsearch.

API Elastic search provided there is a cluster API, search API, API index, a document API, cat API, API aggregation and so on, this paper explains Elasticsearch API, using cluster API (hereinafter referred to as cluster API) can view the status of the cluster profiles, cluster status and statistics, cluster node information and so on.

A view cluster health (health)

Use helath API can view the cluster's overall health, when there is abnormal timely warning, for example:

$ curl -X GET "localhost:9200/_cluster/health?pretty"

{
  "cluster_name": "elasticsearch",
  "status": "yellow",
  "timed_out": false,
  "number_of_nodes": 1,
  "number_of_data_nodes": 1,
  "active_primary_shards": 3,
  "active_shards": 3,
  "relocating_shards": 0,
  "initializing_shards": 0,
  "unassigned_shards": 3,
  "delayed_unassigned_shards": 0,
  "number_of_pending_tasks": 0,
  "number_of_in_flight_fetch": 0,
  "task_max_waiting_in_queue_millis": 0,
  "active_shards_percent_as_number": 50.0
}

You can even specify the index to see the index level of health, more convenient to locate the problem.

$ curl -X GET "localhost:9200/_cluster/health/test1,test2?pretty"

1.1 API request parameters

• level

  The value can be Cluster, indices, Shards , used to return information about the different levels of health status, default Cluster .

• wait_for_status

  Value may be green, the state of yellow, red, this parameter represents the cluster becomes a waiting state or a specified time-out return.

• wait_for_no_relocating_shards

  to true / fasle , representing whether the waiting time slice does not exist relocating return the default false.

• wait_for_no_initializing_shards

  to true / fasle , representing whether to wait for initializing a time slice does not exist return the default false.

• wait_for_active_shards

  Value represents the number of active waiting survival time slice, the default is 0, All representatives wait for all the fragments are active in the case of return.

• wait_for_nodes

  Value, waits for a specified number (referred to as N ) returns the node reachable, may **> = N, <= N ,> N, <N **, use may correspond ge (N), le (N ), gt (N), lt (N)

• wait_for_events

  Given a priority queue waiting for all events specified priority return after treatment. Priority may be assigned immediate, urgent, high, normal, low, languid

• timeout

  When wait_for_XXX parameters specified timeout period, the default is 30s

• master_timeout

  Master node connection timeout, if not set, consistent with the timeout.

• local

  true / false, true representative to obtain corresponding information from the local node, obtaining information representative of false from the master node, the default is false.

1.2 API response parameter

The results shown above, following the return value Important fields are explained in detail:

• cluster_name

  Name of the cluster, each cluster has a unique name.

• status

  Status of a cluster, all primary into green, yellow, red three kinds, Green and replica representative of slice cluster fragments are operating normally, yellow master slice represents all are operating normally, but the presence of a normal copy of unallocated patch , red representatives master slice allocation was not successful.

• timed_out

  Whether this query timeout value of true / false.

• number_of_nodes

  The number of cluster nodes have.

• number_of_data_nodes

  The number of data nodes in the cluster have.

• active_primary_shards

  The main active fragment

• active_shards

  Activities of fragmentation

• relocating_shards

  Displays the current number of slices are moved from one node to other nodes. In general, it should be 0, but when Elasticsearch discovered cluster is not balanced, the value will rise. For example: add a new node, or a node is down.

• initializing_shards

  Is the number of slices you just created. For example, if you just created the first index, fragmentation will be short-lived in the initializingstate. This usually is a temporary event, fragmentation should not be long-term stay in the initializingstate. You may also see just reboot the node when initializingfragmentation: When the slice is loaded from disk, they will from the initializingstart state.

• unassigined_shards

  Already exist in the cluster state of fragmentation, but actually in the cluster and found none. Unassigned generally fragmented copy source is unassigned. For example, an index of 5 and 1 copy of the fragment, a single node in the cluster, there will be five copies of unassigned fragmented. If your cluster is a redstate will retain long-term unallocated fragmented (due to lack of primary fragmentation)

• number_of_pending_tasks

  The number of tasks waiting in

• number_of_in_flight_fetch

  The number of tasks carried out in

• task_max_waiting_in_queue_millis

  The maximum time job waiting in the queue.

• active_shards_percent_as_number

  Fragmentation percentage of all active slices.

Second, check the cluster status

Cluster state API provides a comprehensive view of the method of the cluster state information, in response to many of the returned content, including maps, nodes, and other document metadata, elasticsearch filter is provided, the corresponding content can be acquired as needed.

$ curl -X GET "localhost:9200/_cluster/state?pretty"

Note: When the build process also cluster, in response to the cluster_uuidvalue of the field may be _na_, the state of the cluster version may be -1

By default, the cluster status request is routed to the primary node to ensure that returns the latest cluster state. For debugging purposes, you can add local = true through the query string to retrieve the status of a particular local cluster nodes.

$ curl -X GET "localhost:9200/_cluster/state/{metrics}/{indices}?pretty"

• It refers to the index of indices, separated by commas

• The following values ​​are selected metrics, a plurality of values ​​separated by commas

  • version

    Status of the cluster version

  • master_node

    Information master_node segment, on the primary node cluster

  • nodes

    Segment information nodes, the nodes on the cluster

  • routing_table

    Information routing_table segment

  • metadata

    Metadata segment information, regarding the configuration, mapping metadata information

  • blocks

    Segment information blocks

  • _all

    Including all information

Third, view the cluster statistics

Various statistical summary statistics for the cluster API is used to view the cluster, including CPU, memory, storage, documents, statistics and other information.

$ curl -X GET "localhost:9200/_cluster/stats?human&pretty"

{
  "_nodes" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "nhG2-6K6RU2kDreqsUZy6A",
  "timestamp" : 1570199646793,
  "status" : "yellow",
  "indices" : {
    "count" : 1,
    "shards" : {
      "total" : 3,
      "primaries" : 3,
      "replication" : 0.0,
      "index" : {
        "shards" : {
          "min" : 3,
          "max" : 3,
          "avg" : 3.0
        },
        "primaries" : {
          "min" : 3,
          "max" : 3,
          "avg" : 3.0
        },
        "replication" : {
          "min" : 0.0,
          "max" : 0.0,
          "avg" : 0.0
        }
      }
    },
    "docs" : {
      "count" : 0,
      "deleted" : 0
    },
    "store" : {
      "size" : "783b",
      "size_in_bytes" : 783
    },
    "fielddata" : {
      "memory_size" : "0b",
      "memory_size_in_bytes" : 0,
      "evictions" : 0
    },
    "query_cache" : {
      "memory_size" : "0b",
      "memory_size_in_bytes" : 0,
      "total_count" : 0,
      "hit_count" : 0,
      "miss_count" : 0,
      "cache_size" : 0,
      "cache_count" : 0,
      "evictions" : 0
    },
    "completion" : {
      "size" : "0b",
      "size_in_bytes" : 0
    },
    "segments" : {
      "count" : 0,
      "memory" : "0b",
      "memory_in_bytes" : 0,
      "terms_memory" : "0b",
      "terms_memory_in_bytes" : 0,
      "stored_fields_memory" : "0b",
      "stored_fields_memory_in_bytes" : 0,
      "term_vectors_memory" : "0b",
      "term_vectors_memory_in_bytes" : 0,
      "norms_memory" : "0b",
      "norms_memory_in_bytes" : 0,
      "points_memory" : "0b",
      "points_memory_in_bytes" : 0,
      "doc_values_memory" : "0b",
      "doc_values_memory_in_bytes" : 0,
      "index_writer_memory" : "0b",
      "index_writer_memory_in_bytes" : 0,
      "version_map_memory" : "0b",
      "version_map_memory_in_bytes" : 0,
      "fixed_bit_set" : "0b",
      "fixed_bit_set_memory_in_bytes" : 0,
      "max_unsafe_auto_id_timestamp" : -1,
      "file_sizes" : { }
    }
  },
  "nodes" : {
    "count" : {
      "total" : 1,
      "data" : 1,
      "coordinating_only" : 0,
      "master" : 1,
      "ingest" : 1
    },
    "versions" : [
      "6.5.4"
    ],
    "os" : {
      "available_processors" : 4,
      "allocated_processors" : 4,
      "names" : [
        {
          "name" : "Windows 10",
          "count" : 1
        }
      ],
      "mem" : {
        "total" : "7.8gb",
        "total_in_bytes" : 8482287616,
        "free" : "2.5gb",
        "free_in_bytes" : 2716463104,
        "used" : "5.3gb",
        "used_in_bytes" : 5765824512,
        "free_percent" : 32,
        "used_percent" : 68
      }
    },
    "process" : {
      "cpu" : {
        "percent" : 0
      },
      "open_file_descriptors" : {
        "min" : -1,
        "max" : -1,
        "avg" : 0
      }
    },
    "jvm" : {
      "max_uptime" : "3.4d",
      "max_uptime_in_millis" : 294220612,
      "versions" : [
        {
          "version" : "1.8.0_221",
          "vm_name" : "Java HotSpot(TM) 64-Bit Server VM",
          "vm_version" : "25.221-b11",
          "vm_vendor" : "Oracle Corporation",
          "count" : 1
        }
      ],
      "mem" : {
        "heap_used" : "238.8mb",
        "heap_used_in_bytes" : 250430752,
        "heap_max" : "1.9gb",
        "heap_max_in_bytes" : 2112618496
      },
      "threads" : 46
    },
    "fs" : {
      "total" : "88.1gb",
      "total_in_bytes" : 94684311552,
      "free" : "18.4gb",
      "free_in_bytes" : 19831750656,
      "available" : "18.4gb",
      "available_in_bytes" : 19831750656
    },
    "plugins" : [ ],
    "network_types" : {
      "transport_types" : {
        "netty4" : 1
      },
      "http_types" : {
        "netty4" : 1
      }
    }
  }
}

You may also be used Node Filter filter content filtering. Content on Node Filter, please refer to the following chapters.

Fourth, the task waiting for clusters

This API returns the cluster-level changes have been submitted but not executed, for example, create an index, updated maps, films and other distribution points, under normal circumstances should be empty.

$ curl -X GET "localhost:9200/_cluster/pending_tasks?pretty"

Fifth, the cluster re-routing

reroute command allows you to manually change the assignment of each fragment cluster. For example, debris can be explicitly moved from one node to another, can cancel the assignment can explicitly debris unallocated assigned to a particular node.

curl -X POST "localhost:9200/_cluster/reroute?pretty" -H 'Content-Type: application/json' -d'
{
    "commands" : [
        {
            "move" : {
                "index" : "test", "shard" : 0,
                "from_node" : "node1", "to_node" : "node2"
            }
        },
        {
          "allocate_replica" : {
                "index" : "test", "shard" : 1,
                "node" : "node3"
          }
        }
    ]
}
'

• When routing.rebalance.enable set to true, then after reroute, Elasticsearch will be re-balanced cluster.
• When cluster.routing.allocation set to false, the cluster fragmentation is disabled distribution, using only manually assigned reroute.

command parameters are:

• move

  A slice for the activity moves from one node to another node. Support of the request body parameters are:

  • index

    Name of the index

  • shard

    Slice number

  • from_node

    The source node

  • to_node

    Target node

• cancel

  Unassigning a slice of default fragments can only cancel a copy of fragmentation, if you want to cancel the allocation of primary fragmentation, you need to specify allow_primaryparameters true. Support of the request body parameters are:

  • index

    Name of the index

  • shard

    Slice number

  • allow_primary

    to true / to false , whether to allow the deallocation on behalf of the master slice.

• allocate_replica

  Allocating a slice is not allocated to a particular node. Supported parameters are as follows:

  • index

    Name of the index

  • shard

    Slice number

  • node

    Name of the node

Sixth, update the cluster configuration

6.1 To obtain the cluster configuration

$ curl -X GET "localhost:9200/_cluster/settings?pretty"

6.2 update cluster configuration

After the update is divided into persistent, interim update cluster configuration, the cluster configuration updates persistence restart still in force, the interim update does not take effect after a cluster restart.

Update persistence configuration

$ curl -X PUT "localhost:9200/_cluster/settings?pretty" -H 'Content-Type: application/json' -d'
{
    "persistent" : {
        "indices.recovery.max_bytes_per_sec" : "50mb"
    }
}
'

---

Update temporary configuration

$ curl -X PUT "localhost:9200/_cluster/settings?flat_settings=true&pretty" -H 'Content-Type: application/json' -d'
{
    "transient" : {
        "indices.recovery.max_bytes_per_sec" : "20mb"
    }
}
'

6.3 Reset cluster configuration

Way configurations only need to reset the cluster configuration item is set to null

$ curl -X PUT "localhost:9200/_cluster/settings?pretty" -H 'Content-Type: application/json' -d'
{
    "transient" : {
        "indices.recovery.max_bytes_per_sec" : null
    }
}
'

6.4 reset cluster configuration using wildcards

$ curl -X PUT "localhost:9200/_cluster/settings?pretty" -H 'Content-Type: application/json' -d'
{
    "transient" : {
        "indices.recovery.*" : null
    }
}
'

6.5 cluster configuration order

When simultaneously using the configuration file, configuration API configuration updates, the effectiveness of the following sequence:

1. Temporary cluster configuration

2. Persistence cluster configuration

3. elasticsearch.yml profile configuration

General recommendations put elasticsearch.yml local configuration profiles, cluster related configuration uses settings API.

Seven, get the cluster configuration

7.1 Get set explicitly configure

$ curl -X GET "localhost:9200/_cluster/settings?pretty"

7.2 Get the default configuration

$ curl -X GET "localhost:9200/_cluster/settings?include_defaults=true&pretty"