The mechanism for processing data is called a pipeline. At the configuration level, pipelines describe the coupling between data sources and corresponding sinks for transforming and publishing data. This functionality is handled by the notification agent.
source is the producer of the data: samplesor events. Essentially, it's a set of notification handlers that emit data points for a matching set of meters and event types.
Each source configuration encapsulates name matching and mapping into one or more sinks for publication.
On the other hand, sinks are consumers of data, providing logic for transforming and publishing data from related sources.
In fact, sink describes a series of handlers. The chain starts with zero or more transformers and ends with one or more publishers. The first transformer in the chain passes the data from the corresponding source, takes some action, such as deriving the rate of change, performing a unit conversion or aggregation, and then publishing .
pipeline configuration
The notification agent supports two pipelines: one for samples and one for events. Pipelines can be enabled and disabled by setting pipeline options in [notifications].
By default, the actual configuration for each pipeline is stored in separate configuration files: pipeline.yamland event_pipeline.yaml. The location of the configuration file can be set by the pipeline_cfg_fileand event_pipeline_cfg_fileoptions, see the configuration file template: Ceilometer Configuration Options .
The definition of the meter pipeline is as follows:
---
sources:
- name: 'source name'
meters:
- 'meter filter'
sinks:
- 'sink name'
sinks:
- name: 'sink name'
transformers: 'definition of transformers'
publishers:
- 'list of publishers'
There are several ways to define a meter list for a pipeline source. A list of valid meters can be found in Measurements . There is a way to define all the meters or only include or filter some of the meters, a source configuration operation should be as follows:
- Use the * wildcard to include all meters. But it's wise to only choose the meters you intend to use to avoid flooding the metering database with unused data.
- To define a list of meters, use any of the following:
To include partial meters, use the meter_namesyntax .
To filter partial meters, use the !meter_namesyntax .
Note: The OpenStack Telemetry Service does not have any duplication checking between pipelines, if you add a meter in multiple pipelines, the duplication is assumed to be intentional and can be stored multiple times depending on the specified sink.
The above definition method can be used for the following combinations:
- Only use wildcard symbols.
- 使用 included meters。
- 使用 excluded meters。
- Wildcards are used in conjunction with excluded.
Note: At least one of the above changes should be included in the meters section. Included and excluded meters cannot coexist in the same pipeline. Wildcards and included meters cannot coexist in the same pipe definition section.
The transformers section of the pipeline sink provides the possibility to add a list of transformers definitions. Existing transformers:
| Transformer name | Configured reference name |
|---|---|
| Accumulator | accumulator |
| Aggregator | aggregator |
| Arithmetic | arithmetic |
| Rate of change | rate_of_change |
| Unit conversion | unit_conversion |
| Delta | delta |
The publishers section contains a list of publishers where sample data should be sent after possible transformations.
Similarly, the event pipeline definition looks like this:
---
sources:
- name: 'source name'
events:
- 'event filter'
sinks:
- 'sink name'
sinks:
- name: 'sink name'
publishers:
- 'list of publishers'
Event filters use the same filtering logic as the meter pipeline.
Transformers
Note: Transformers hold data in memory, so there is no guarantee of persistence in certain scenarios. Using a solution like Gnocchi allows for a more durable and efficient solution.
A converter definition can contain the following fields:
- the name of the name converter
- parameters converter parameters
The parameters section can contain transformer specific fields, in the case of rate of change, fields like source and target contain different subfields, depending on the transformer implementation.
The following are supported transformers:
Rate of change transformer
This type of converter calculates the time change between two data points. The following transformer example defines the cpu_utilmeter:
transformers:
- name: "rate_of_change"
parameters:
target:
name: "cpu_util"
unit: "%"
type: "gauge"
scale: "100.0 / (10**9 * (resource_metadata.cpu_number or 1))"
The rate-of-change transformer generates the cpu_utilmeter from the sample value of the cpu counter, which represents the cumulative CPU time in nanoseconds. The above transformer definition defines a scaling factor (for nanoseconds and multi-cpu) that is applied before transformation to derive a sequence of metric samples with % units from the sequential values of the cpu table.
The definition of disk I/O rate, also generated by the rate-of-change converter:
transformers:
- name: "rate_of_change"
parameters:
source:
map_from:
name: "disk\\.(read|write)\\.(bytes|requests)"
unit: "(B|request)"
target:
map_to:
name: "disk.\\1.\\2.rate"
unit: "\\1/s"
type: "gauge"
Unit conversion transformer
This type of converter is used for unit conversion. It takes the volume of the meter and multiplies it by the given scale expression. Also supports fields like transformer rate of change with map_fromand .map_to
Sample configuration:
transformers:
- name: "unit_conversion"
parameters:
target:
name: "disk.kilobytes"
unit: "KB"
scale: "volume * 1.0 / 1024.0"
Use map_fromand map_to:
transformers:
- name: "unit_conversion"
parameters:
source:
map_from:
name: "disk\\.(read|write)\\.bytes"
target:
map_to:
name: "disk.\\1.kilobytes"
scale: "volume * 1.0 / 1024.0"
unit: "KB"
Aggregator transformer
Such a converter aggregates incoming samples until enough samples are reached or a timeout occurs.
A timeout can be specified with the retention_timeoption . Specify the parameter size if you want to refresh the aggregation after a certain number of samples have been aggregated.
The created sample size is the sum of the sample sizes input to the 1-transformer. Samples can be aggregated by project_id, user_idand resource_metadataattributes . Aggregate based on selected attributes, specify them in the configuration, and set the value of that attribute to get new samples (first use the first sample attribute, last take the last sample attribute, then delete the attribute).
By resource_metadataaggregating 60 seconds of sample values and saving the latest received sample resource_metadata:
transformers:
- name: "aggregator"
parameters:
retention_time: 60
resource_metadata: last
Aggregate each 15 samples by user_idand resource_metadata, and keep the first received sample user_id, and delete resource_metadata:
transformers:
- name: "aggregator"
parameters:
size: 15
user_id: first
resource_metadata: drop
Accumulator transformer
This converter simply caches samples until enough samples are reached, then flushes them all down the pipeline at once:
transformers:
- name: "accumulator"
parameters:
size: 15
Multi meter arithmetic transformer
Such converters allow us to perform arithmetic operations on one or more meters, and/or metadata. E.g:
memory_util = 100 * memory.usage / memory
A new sample will be created according to the properties described in the targetsection . The sample size is calculated based on the provided expression. Calculations are performed on samples of the same resource.
Note: The scope of calculation is limited to meters within the same interval.
Example configuration file:
transformers:
- name: "arithmetic"
parameters:
target:
name: "memory_util"
unit: "%"
type: "gauge"
expr: "100 * $(memory.usage) / $(memory)"
To demonstrate the use of metadata, the following implementation of a novel measurer shows the average CPU time per core:
transformers:
- name: "arithmetic"
parameters:
target:
name: "avg_cpu_per_core"
unit: "ns"
type: "cumulative"
expr: "$(cpu) / ($(cpu).resource_metadata.cpu_number or 1)"
Remarks: Expression evaluation gracefully handles NaNs and exceptions. In this case, it does not create a new sample, but only logs a warning.
Delta transformer
This transformer computes the change between two sample data points for a resource. It can be configured to capture only positive growing deltas.
Example configuration:
transformers:
- name: "delta"
parameters:
target:
name: "cpu.delta"
growth_only: True
Publishers
The Telemetry Service provides several transmission methods to transmit the collected data to external systems. Consumers of this data vary widely, as with surveillance systems, data loss is acceptable but billing systems require reliable data transfer. Telemetry provides a means to meet both system requirements.
Publisher components can persist data to persistent storage or send it to one or more external consumers via the message bus. A chain can contain multiple publishers.
To address this, multiple publishers can be configured for each data point in the telemetry service, allowing the same technical meter or event to be published multiple times to multiple destinations, each potentially using a different transport.
The following publisher types are supported:
gnocchi (default)
When a gnocchi publisher is enabled, metrics and resource information are pushed to gnocchi for time series optimized storage. Gnocchi must be registered with the Identity service, because Ceilometer uses the Identity service to discover the exact path.
More details on how to enable and configure gnocchi can be found on its official documentation page .
panko
Event data in cloud computing can be stored in panko, which provides an HTTP REST interface to query system events in OpenStack. Push data to panko, set publisher to panko://.
notifier
The notifier publisher can be specified in notifier://?option1=value1&option2=value2the form . It uses oslo.messaging to emit AMQP data. Any consumer can then subscribe to the published topic for additional processing.
The following customization options are available:
per_meter_topic
The value of this parameter is 1. It is used to publish samples on additional metering_topic.sample_nametopic default queue.metering_topic
policy
Used to configure the behavior of the case, when the publisher fails to send the sample, the possible predefined values are:
default : used to wait and block until samples are sent.
drop: Used to drop samples that could not be sent.
queue: used to create an in-memory queue and retry to send samples to the queue during the next sample release (queue length can be configured with the max_queue_lengthproperty , default value is 1024).
topic
Topic name of the queue to publish to. Setting this option will metering_topicoverride event_topicthe theme set by and by default. This option can be used to support multiple consumers.
udp
This publisher can be specified in udp://<host>:<port>/the form . It sends out metering data over UDP.
file
This publisher can be specified in file://path?option1=value1&option2=value2the form . This publisher logs measurement data to a file.
Note: If no filename and location are specified, the filepublisher will not log any meters, but instead log a warning message in the configured log file for Telemetry.
The following options are available to file publishers:
max_bytesWhen this option is greater than zero, it will cause a rollover. When the specified size is about to overflow, the file will be closed and a new file will be silently opened for output. If its value is zero, then the rollover will not happen.
backup_countIf the value is non-zero, extensions will be appended to the old log's filename, such as .1, .2, and so on, until the specified value is reached. Writing status and files containing the latest data are always without any specified extension.
http
The Telemetry Service supports sending samples to external HTTP targets. The samples are issued without any modification. To set this option as a notification agent target, http://set to the publisher endpoint in the pipeline definition file. The HTTP target should be set with the publisher declaration. For example, additional configuration options can http://localhost:80/?option1=value1&option2=value2be .
The following options are available:
timeoutThe number of seconds before the HTTP request times out.
max_retriesThe number of times to retry the request before failing.
batchIf false, the publisher will send each sample and event separately, regardless of whether the notification agent is configured for batch processing.
verify_sslIf false, disables ssl certificate verification.
The default publisher is gnocchi, no other options are specified. /etc/ceilometer/pipeline.yamlThe sample publisher section in the configuration file looks like this:
publishers:
- gnocchi://
- panko: //
- udp://10.0.0.2:1234
- notifier://?policy=drop&max_queue_length=512&topic=custom_target
pipe split
Note: Partitioning is only necessary if the pipeline contains transformations. With the support of some publishers, it has secondary benefits. Under heavy workloads, multiple notification agents can be deployed to handle the flood of incoming messages from the monitoring service. If transformations are enabled in the pipeline, notification brokers must be coordinated to ensure related messages are routed to the same broker. To enable coordination notification, set the workload_partitioningvalue in the section.
To distribute messages across brokers, the pipeline_processing_queuesoption . This value defines how many pipeline queues to create and then distribute to active notification agents. It is recommended that the number of processing queues at least match the number of agents.
Increasing the number of processing queues will improve the distribution of messages among brokers. It also helps to minimize requests to the Gnocchi storage backend. It will also increase the load on the message queue as it will use the queue to fragment data.
Warning: Reducing the number of processing queues may result in data loss, as previously created queues may no longer be assigned to active agents. It is only recommended that you increase the processing queue.
Reprint: http://luozn.top/2018/04/11/223/