Understand ECS (Elastic Common Schema)

• Elastic Common Schema

  The Elastic Common Schema (ECS) is an open source specification, developed with support from the Elastic user community.

  ECS defines a common set of fields to be used when storing event data in Elasticsearch, such as logs and metrics. ECS specifies field names and Elasticsearch datatypes for each field, and provides descriptions and example usage.

  The goal of ECS is to enable and encourage users of Elasticsearch to normalize their event data, so that they can better analyze, visualze, and correlate the data represented in their events.

  ECS enables and encourages users to normalize event data in order to better analyze, visualize, and correlate their events. Collected events can be normalized at ingest time, consistently searched across indices, and visualized predictably.

• ECS Field Levels

  ECS defines “Core” and “Extended” fields:

  • Core fields. Fields that are most common across all use cases are core fields.
  • Extended fields. Any field that is not a core field is defined as an extended field.

  Each ECS field in a table is identified as core or extended.

• General Guidelines

  The document MUST have the @timestamp field.

  Use the data types defined for an ECS field.

  Use the ecs.version field to define which version of ECS is used.

  Map as many fields as possible to ECS.

  Field names must be lower case.

  Comebine words using underscore.

  No special characters except underscore.

  Use present tense unless field describes historical information.

  Use singluare and plural names properly to reflect the field content.

  Use prefixes for all fields, except for the base fields. Such a grouping is called a field set.

  Nest fields inside a field set with dots.

  General to specific. Organise the nesting of field sets from general to specific, to allow grouping fields into objects with a prefix like host.*.

  Avoid repetition or stuttering of words.

  Avoid abbrevitaions when possible.