Allocation Service
1. basic concepts
Heat is a service to orchestrate complex cloud applications based on a template. It currently supports Amazon CloudFormation template formats, and also supports its own Hot Heat template format. Use templates to simplify the complexity of defining and deploying infrastructure, services and applications. Template supports a rich resource types, covering not only the common infrastructure, including computing, networking, storage, mirroring, also covers advanced alarm Ceilometer of resources, Sahara cluster instance Trove and so on. It can be implemented based on the template cloud environment initialization resource dependency treatment, the basic operation of deployment, advanced features can be resolved automatically shrink, load balancing and so on. Hot Heat currently own template format is constantly improving, while also supporting AWS CloudFormation template (CFN). Below we will explain in detail Hot template.
Heat service includes the following important components.
① Heat-api components to achieve natural OpenStack support REST API. The assembly by the request is transmitted via the API to AMQP Heat engine to process the API requests.
② Heat-api-cfn compatible AWS CloudFormation assembly provides the API, the API will also forwards the request to a heat engine via AMQP.
③ Heat-engine components provide the Heat leading collaboration capabilities.
First Horizon or user command to submit the request contains the template and input parameters, command-line tools or Horizon will request into a format REST API calls and Heat-api, or Heat-api-cfn. Heat-api and then Heat-api-cfn will verify the correctness of the template, for processing the request to the Heat Engine AMQP transferred through asynchronous, as shown in FIG.
When Heat Engine get request, the request will be resolved to the various types of resources, each resource corresponds OpenStack other services clients, and other services via the request to the REST. By thus analysis and collaboration, final completion processing the request.
Here, the role of Heat Engine is divided into three layers: layer 1 processing request Heat level, Stack is created according to a template and the input parameters, where Stack made by a combination of various resources. Layer Stack second resolution in dependence of resources, and nested relationship of Stack Stack. Tier 3 is based on the relationship parsed sequentially invoke various services client to create a variety of resources, as shown.
2 . Arrange
Arrangement, by definition, are sequentially arranged according to a certain object. In the IT world, a complete arrangement generally comprises a server disposed on the machine, installed CPU, memory, hard drives, power, into the network interface, install the operating system, configure the operating system, middleware installed, middleware configure, install applications, and configure application publisher. For complex applications that require deployment on multiple servers, you need to repeat this process, and the need for coordination to configure each application module. For example, the foregoing configuration application server connected to the database server behind. Figure 8-3 shows a typical application requires orchestration project.
In the cloud computing world, this layer becomes virtual machine's VM or container. Each resource element and the operating system itself has become the focus of IaaS management VM orchestration needed. After the operating system installation configurations are also covered by the scope of the Arrangement IaaS. In addition, the frame can provide access to PaaS and IaaS SaaS arrangement also the arrangement range.
Application projects choreography diagram:
3 . Heat choreography
OpenStack Horizon command line and provides a way for users to resource management, but the efficiency of these two methods is not high. Even if the command line is saved as a script, input / output, dependent still need to write additional scripts relationship between maintenance, and easy to expand. If the user directly through the REST API to write programs, it will also lead to additional complexity. Therefore, these two approaches are not conducive to the user to batch resource management and orchestration of resources through OpenStack.
Heat came into being in this case, it uses the industry-popular template approach to design and define the layout. Users only need to open a text editor, write a Key-Value based template, you can easily get the desired layout. For the convenience of users, Heat provides a number of template examples, usually the user need only select the layout you want, by copying, pasting way to complete the preparation of the template.
Heat the choreography follows. First, OpenStack provide their own infrastructure resources, including computing, networking and storage resources. By orchestrating these resources, users can get the most basic VM. In addition, in the process of scheduling VM, the user can write simple scripts to make simple configuration for VM. Then, the user may provide Software Configuration Heat Software Deployment and other complex configuration of the VM, such as installing software and configuration software. Secondly, when the user needs some advanced features, such as a need to be able to set auto-scaling based on the load of the VM group, or a set of load-balanced VM, Heat Load Balance and other offers AutoScaling and template support. Heat only in the length of the period of Template, these complex applications can be realized. Heat, such as support for complex applications such as AutoScaling and Load Blance has been very mature, there are a variety of templates available for reference.
4 . Heat template
Heat template currently supports two formats, one is based on CFN template JSON format, and the other one is based HOT template YAML format. CFN template mainly in order to maintain compatibility with the AWS. HOT own template is Heat, resource type richer, better reflect the characteristics of the Heat template.
A typical HOT template consists of the following elements.
(1) template version
Required fields corresponding to the specified template version, Heat will be tested depending on the version.
(2) the list of parameters
Optional, refers to the input parameter list.
(3) Resource List
Required, refers to a variety of resources generated Stack contains. Can define dependencies between resources, such as generating Port, and then port then generates VM.
(4) Output List
Optional, refers to information generated Stack exposed, can be used to the user, it may be used to provide an input to other Stack.
Examples are as follows.
heat_template_version: 2013-05-23 // Version information
parameters: // custom variable
ImageID:
type: string
resources: // resources described, such as network, Example
server1:
type: OS :: :: New Server
outputs: // return value
server1_private_ip:
value: { get_attr: [ server1, first_address ] }
For different resources, Heat provides the corresponding resource types. Heat can be queried by the command.
# heat resource-type-list
For example, for a VM, Heat provides OS :: Nova :: Server command. OS :: Nova :: Server there are some parameters, such as key, image and flavor, these parameters can be specified directly, may be provided by the customer when creating Stack, can also be obtained by other parameters contexts.
Examples are as follows.
// Heat resource usage and parameter settings
resources:
private_net:
type: OS::Neutron::Net
properties:
name: private-net
private_subnet:
type: OS::Neutron::Subnet
properties:
network_id: { get_resource: private_net }
cidr: 172.16.2.0/24
gateway_ip: 172.16.2.1
For the parameters in the form of properties in order to network, for example, can be obtained from the existing network.
# Neutron subnet-show IT-Subnet // View Details IT-subnet subnet
Heat and Ceilometer service can achieve automatic retractable resource allocation, it can achieve load balancing resource allocation. Future Heat do more important functions in the Openstack, for example, so-called elastic expansion. Holds many lessons for the most popular App Store, but also by the Heat to achieve.
1. Operation and maintenance of base Heat
(1) use the stack to create a template test-stack.yml stack of named Orchestration
And command execution results are as follows.
#openstack stack create -t test-stack1.yml --parameter NetID=int-net test
or:
# heat stack-create orchestration -f test-stack.yml --parameters "ImageID=centos;NetID=int-net"
(2) looking at a stack list.
# heat stack-list
For more information (3) View stack
# heat stack-show orchestration
(4) delete the stack
# heat stack-delete orchestration
(5) look at the stack resource list
# heat resource-list orchestration
(6) View stack resources
# heat resource-show orchestration server1
(7) Check the output list
# heat output-list orchestration
(8) to view the output value
# heat output-show orchestration server1_private_ip
(9) View event list
# heat event-list orchestration
(10) View Resource Event Details
# heat event-show orchestration server1 《ID》（de878b64-7015-4d31-ad74-0b69d47710a0）
Monitoring Service
In general, cloud computing platform in addition to providing services, but also served as a "Metering, Billing" and other functions. This section will detail cloud measurement services Ceilometer.
1 . basic concepts
Meaning (1) Ceilometer of
In general, cloud services, especially public cloud services, in addition to providing basic services, but also bear the "billing" function, there are three levels of public cloud billing terms.
① Measurement (Metering): resources used for data collection, which includes the use of data objects, the user, time and amount.
② Accounting (Rating): The resource usage data into billable in accordance with business rules and project costing.
③ billing (Billing): billing money.
Ceilometer goal is Metering terms of metering, billing for the upper, settlement or monitoring application resource usage data collection function provides unified.
(2) Ceilometer main concepts
The main concept of the following five categories Ceilometer.
① Meter: resource measurement items used, its properties include the name (name), a unit (Unit), type (cumulative: integrated value, delta: change value, gauge: discrete or fluctuation value) and the resource attributes corresponding.
② Sample: is a moment of a resource (resource) of a meter value. Sample collection interval has concept, i.e., the data collection time interval. Meter In addition to the properties, as well as timestampe (sampling time) and Volume (sample values) properties.
③ Statistics: a period of time samples within the aggregate value (Period), includes a count (Count), the maximum (Max), the minimum (Min), average (Avg), summation (the Sum) and the like.
④ Resource: refers to the resource object being monitored can be a virtual machine, a physical machine or a cloud hard drive.
⑤ Alarm: the alarm mechanism is Ceilometer can alarm threshold or by a combination of the condition, action, and is triggered when the alarm setting.
(3) Alarm state
ALARM （告警状态）：
{"current": "alarm", "alarm_id": "742873f0-97f0-4d99-87da-b5f7c7829b7f", "reason": "Remaining as alarm due to 1 samples outside threshold, most recent: 0.138333333333", "previous": "alarm"}
正常状态 （数据充足，未告警）：
{"current": "ok", "alarm_id": "742873f0-97f0-4d99-87da-b5f7c7829b7f", "reason": "Remaining as ok due to 1 samples inside threshold, most recent: 0.138333333333", "previous": "ok"}
Insufficient Data （默认状态 - 数据不足）：
{"current": "insufficient data", "alarm_id": "742873f0-97f0-4d99-87da- b5f7c7829b7f", "reason": "1 datapoints are unknown", "previous": "ok"}
(4) Alarm operation
Ceilometer currently supports two Action.
'Log:': Alarm Log files are written.
'WebhookURL:': This is a HTTP (S) URL endpoint, e.g. 'http://130.56.250.199: 8080 / alarm / instances_TOO_MANY'. Alarm content will be POST to the URL in the JSON format.
(5) Ceilometer collection mechanism
Ceilometer in each service, the service is associated with the acquisition ceilometer-collector, ceilometer- agent-central, ceilometer-agent-compute and ceilometer-agent-notification. Specific process shown in FIG. Ceilometer collection mechanisms:
Information agent- * Service is responsible for collecting information gathering can publish it in three ways, including RPC, UDP and File. RPC is the information collected distributed in payload manner to the message queue, collector service collected by snoop queue information, and stored in the storage medium; UDP create a UDP channel through socket, then the collector receives through bind the socket data, and stored in the storage medium; file more direct way, the data is collected in a manner filelog write the log file.
As for which way to publish to use, depending on how the pipeline configuration file, you can view /etc/ceilometer/pipeline.yaml specific configuration of publishers.
agent- * acquisition component responsible for three different types of information collected, agent-notification pushed to collect the various components oslo-messaging message, oslo-messaging are the message queue OpenStack entire frame, all the components of the message queue uses this component ; agent-compute only responsible for collecting information on CPU memory and IO and other virtual machines, so needs to be installed on the machine Hypervisor; agent-central is to collect useful information through the various components of the API method. Simply listen AMQP agent-notification queue can receive the information, and the agent-compute and agent-central require periodic polling to collect information, as shown in FIG. Polling collection:
Information collected by the collector through agent- * summary process, ultimately need persisted to storage media, Ceilometer storage currently supported include MySQL, DB2, HBase and MongoDB, from the point of view supported database, monitoring data persistence pressure is still quite big. as the picture shows. Ceilometer collective processing:
(6) Ceilometer data processing
Ceilometer will process the collected data, as shown in FIG processing mechanism, Ceilometer data processing:
① Collect: Meters data collection.
② Transform: Meters data conversion.
③ Publish: Meters data release.
④ Store: Meters data retention.
⑤ Read: Meters data access.
⑥ Alarm: Alarm provided.
Overall architecture. Ceilometer framework:
2 . Meter data processing
Meters Pipeline way process using data, i.e., data sequentially passes Metes (zero or more) and the Transformer (one or more) Publisher, and finally reach the (one or more) Receiver. Wherein, Recivers including Ceilometer Collector and external systems.
Ceilometer publishers meters and transformers is configured to be used according profile /etc/ceilometer/pipeline.yaml. Below cpu meter example to introduce the pipeline configuration.
sources: A source is a producer of samples//对cpu的meters数据进行处理，并进行数据取样
......
- name: cpu_source
interval: 600
meters:
- "cpu"
sinks:
- cpu_sink
......
sinks: A sink on the other hand is a chain of handlers of samples
......
- name: cpu_sink
transformers:
- name: "rate_of_change"
parameters:
target:
name: "cpu_util"
unit: "%"
type: "gauge"
scale: "100.0 / (10**9 * (resource_metadata.cpu_number or1))"
publishers:
- notifier://
This code defines a number of attributes cpu meter.
① interval: 600: Poller cpu samples acquired in interval of 10 minutes.
② cpu meter of transformer is "rate_of_change".
③ cpu meter the publisher is notifier: //, it uses the default configuration data sent through AMQP use oslo.messaging.
Transformer i.e. Sample converter. Common transformer as follows.
① unit_conversion: unit converters, such as temperature to convert from ° F ° C.
② rate_of_change: calculated converter, such as to convert a sample according to a certain calculation rules.
③ accumulator: accumulator, the result of accumulating the converted measurement or iteration.
E.g:
name: "rate_of_change" //数据处理动作为数据改变
parameters:
target:
name: "cpu_util"
unit: "%"
type: "gauge"
scale: "100.0 / (10**9 * (resource_metadata.cpu_number or 1))"Ceilometer data processing:
3 . Publisher dispenser
The following table shows the type Ceilometer Publisher supports.
| Publishers types |
Format |
Description |
Configuration Item |
| Notifier |
notifier://?option1=value1&option2=value2 |
AMQP data samples are sent to the system, and then received Ceilometer collecter. The default AMQP Queue is metering_ topic = metering. This is the default mode |
[publisher_notifier] |
| RPC |
rpc://?option1=value1&option2=value2 |
Similarly with the notifier, also through AMQP, but synchronous operation, so there may be performance issues |
[publisher_rpc] |
| UDP |
udp://<host>:<port>/ |
After UDP port issue. The default UDP port 4952 |
udp_port=4952 |
| File |
file://path?option1=value1&option2=value2 |
Send to save the file |
|
Publisher distributor configured by modifying /ect/ceilometer/ceilometer.conf, items listed below, and a message exemplary centralized configuration of Publisher.
Notifier configuration items below.
[publisher_notifier]
metering_driver = messagingv2
metering_topic = metering
Example:
notifier://?policy=drop&max_queue_length=512
RPC configuration items below.
[publisher_rpc]
metering_topic = metering
Example:
rpc://?per_meter_topic=1
UDP configuration items below.
udp_port=4952
Example:
rpc://?per_meter_topic=1
File configuration items as follows.
file://path?option1=value1&option2=value2
To specify multiple publisher for a meter by configuring /etc/ceilometer/pipeline.yaml. For example, increasing the publisher of a UDP.
sinks:
- name: meter_sink
transformers:
publishers:
- notifier://
- rpc://?per_meter_topic=1
4 . Data retention
After (1) Ceilometer Collector AMQP received from the data unchanged to save it to a specified position by the one or more distributor (dispatchers). as the picture shows. It currently supports the dispatcher as follows.
① file distributor: save to a file.
The distributor is specified by adding a configuration item "dispatcher = file" type of file.
② HTTP dispatcher: save to an external HTTP target.
The distributor is specified by adding a configuration item "dispatcher = http" type HTTP.
③ distributor database: saved to the database.
Add configuration items "dispatcher = database" to specify the distribution type for the database.
(2) Ceilometer Collector supported database types distribution has the following three kinds.
① MongoDB: the default DB.
② SQL DB: support for MySQL, postgreSQL IBM DB2, and so on.
③ HBase DB。
Ceilometer database type:
Ceilometer simultaneously support a plurality of distributor, to save data to a plurality of destination locations. By doing as follows ceilomet-er.conf file and may be used in such database dispatcher simultaneously.
[DEFAULT]
dispatcher = database
dispatcher = file
[dispatcher_file]
backup_count = 5
file_path = /var/log/ceilometer/ceilometer-samples
max_bytes = 100000
5 . Warning
ceilometer-alarm-api use CeilometerREST API acquires statistics data.
ceilometer-alarm-evaluator generates alarm data, and sent through AMQP ceilometer-alarm-notifer.
ceilometer-alarm-notifer the alarm will be sent out by specifying the way. DETAILED process shown in FIG.
Ceilometer alarm mechanism:
1. View
You can view the data required to run a certain time period by the platform GUI or CLI, including network data, instance data, storing data and service resource consumption.
① into the dashboard to find the "administrator" option.
② Open the System Panel, find the "resource usage."
③ into the "Statistics" screen.
④ Select the appropriate measure, grouping value, time, view statistics.
You can use the command to view network data, instance data, storing data and service resource consumption.
View network data and command execution results are as follows.
# ceilometer statistics -m network.incoming.bytes
See Examples of data commands and execution results are as follows.
# ceilometer statistics -m instance
View store data and command execution results are as follows.
# ceilometer statistics -m disk.read.requests
View Service resource consumption and command execution results are as follows.
# ceilometer statistics -m memory
