AWS monitoring services (VI)

AWS CloudWatch

concept

• Content-based monitoring infrastructure components identified
• Send notifications based on a specified index and trigger various operations
• Distributed data and statistics collection system for collecting and tracking metrics.
• By default, seamless collected metrics hypervisor level, such as CPU utilization, the IO byte operation, the network operator byte
• CloudWatch include promoters may trigger termination restart EC2, increase decrease AutoScaling group, send messages to other operations SNS
• Attributes
  • Panels (Dashboards) - can create a custom panel to facilitate the observation of different monitoring objects AWS environment
  • Alarms (Alarms) - When a monitored object exceeds a threshold value, will issue a warning message
  • Events (Events) - reaction carried out for changes that occur in the AWS environment
  • Log (Logs) -Cloudwatch log to help collect, monitor and store log information
• Monitoring indicators
  • Monitoring and support of the vast majority of indicators specified AWS services, including:
    • Auto Scaling，Amazon CloudFront，Amazon CloudSearch，Amazon DynamoDB，Amazon EC2，Amazon EC2容器服务 (Amazon ECS)，Amazon ElastiCache，Amazon Elastic Block Store(Amazon EBS) ，Elastic Load Balancing，Amazon Elastic MapReduce(Amazon EMR)，Amazon Elasticsearch服务， Amazon Kinesis Streams，Amazon Kinesis Firehose，AWS Lambda，Amazon Machine Learning， AWS OpsWorks，Amazon Redshift，Amazon关系数据库服务(Amazon RDS)，Amazon Route 53 ， Amazon SNS，Amazon Simple Queue Service(Amazon SQS)，Amazon S3，AWS Simple Workflow Service(Amazon SWF)，AWS Storage Gateway，AWS WAF和Amazon WorkSpaces。
  • Ability to customize indicators, including:
    • Those indicators application AWS itself invisible, such as page load time, request the error rate, the number of concurrent processes or threads, supported by the API calls PUT indicators
• Monitoring Frequency
  • Basic monitoring every 5 minutes as a data collection point, providing free limited number of indicators and monitoring
  • Detailed monitoring for collection of data points per minute, can be custom indicators, you need to pay to use
  • Support for more fine-grained high-resolution indicators collected every 1s
  • CloudWatch support across the available area and retrieving the polymerization, the polymerization but does not support inter-regional
  • CloudWatch can only monitor performance, can not track changes
• Cloud Design Mode - CloudWatch monitoring software works in conjunction with
  • EC2 CloudWatch not provide internal work, such as operating systems, middleware, applications, etc., which require the use of an independent monitoring system to achieve
  • Can be deployed Nagios, Zabbix, Munin and other software on a separate EC2, AWS to obtain information from the monitoring by CloudWatch API, in order to integrate

CloudWatch Logs

• You can be customized index or CloudWatch Logs to process the data, access to near real-time monitoring logs
• Monitors and logs are stored, in order to help you better understand and running systems and applications
• You can use CloudWatch Logs log data stored in the long-term persistence of high cost-effective storage without having to worry about run out of hard disk space.
• CloudWatch Logs can store individual measurements and other information in a log file
• The default monitoring data retention up to 15 months. And can not be manually deleted
  • Period of less than 60 seconds of data points can be retained for three hours. These data points are high-resolution custom indicators.
  • Period is 60 seconds (1 minute) data points available for 15 days
  • Period is 300 seconds (5 minutes) data points may be retained 63 days
  • Period of 3600 seconds (1 hour) data points may be retained 455 days (15 months)
• Support for real-time monitoring and log files to trigger specific events
• CloudWatch Logs can be used in the following treatment:
  • Real-time streaming data log to the Amazon Kinesis Stream AWS Lambda, or other data processing solutions
  • S3 to store or archive in batch form in Glacier
  • Administrators can see through the console
• CLoudWatch Agent
  • In EC2 Linux systems can collect EC2 within the system logs by installing CloudWatch log Agent
  • Amazon Linux, Ubuntu, CentOS, Red Hat Enterprise Linux and Windows support
• CloudWatch Logs Insights
  • One for CloudWatch Logs of pay-as interactive integrated log analysis capabilities. It allows developers, operators and system engineers, search and visualize their logs, to help them understand, improve and debug their applications.

    Alert

• You can create an alert in the account to monitor any Amazon CloudWatch indicators. For example, you can create an alert to monitor the CPU usage of Amazon EC2 instances, Amazon ELB request delays, Amazon DynamoDB tables throughput, Amazon SQS queue length, even AWS bill costs.
• You can also specific to the custom metric custom applications or infrastructure to create an alert. If the custom indicator is the high-resolution metrics, you can choose to create a high-resolution alarm, the alarm will alert at the time of 10 seconds or 30 seconds period.
• When creating an alert may be configured to perform one or more automatic operation exceeds a defined threshold in the selected monitoring index. For example, you can set alerts to send e-mail, posted to the SQS queue, stop or terminate a Amazon EC2 instance, or Auto Scaling execution policy. Because Amazon CloudWatch alarms with Amazon Simple Notification Service to achieve integration, and therefore can also be used to inform any type supported by the SNS.
• Alert History is valid for 14 days

Dashboard

• Amazon CloudWatch control panel allows you to create, customize, and interact with AWS resources and save custom chart indicators.
• Automation control panel pre-built best practices recommended by AWS services, the resources of perception, and can be dynamically updated to reflect the latest status of key performance indicators. You can now filter and failure to exclude a particular view, without the need to add additional code to reflect the latest state of the AWS resources. After determining the root cause of performance problems, you can go directly to the AWS resources to act quickly.
• It will automatically refresh when the control panel is open.

Event

• Amazon CloudWatch Events (CWE) is a system event stream AWS resources to change the description.
• When an event matches a rule that you create in the system, you can automatically call an AWS Lambda function, the event will be relayed to the Amazon Kinesis flow, Amazon SNS topic to send notifications, or to call a built-in workflows.
• Event is not the same as AWS Config check compliance, nor CloudTrail as record calls record.

CloudWatch restrictions

• Each AWS account to save up to 5000 alarm
• The default monitoring collection and aggregation size of 1 minute
• Index data collected by default for 15 days, or longer retention needs to dump S3 in Glacier
• The default does not support monitoring of internal memory and system indicators, need custom configuration

AWS CloudTrail

Outline

• CloudTrail user activities can be provided by recording operations performed on account of visibility. CloudTrail can record information for each operation is important, including a request originator, service usage, the operation is performed, operating parameters, and AWS service returns a response element. This information can help you keep track of changing circumstances AWS resources to help you troubleshoot operational problems. CloudTrail easy for you to ensure compliance with internal policies and regulatory standards together.
• An event included activities related to information: the requesting party service use, operation performed, parameters of the operation, as well as AWS service returns a response element.
• AWS account does capture a variety of operations, including the AWS API calls and related events, and log files uploaded to S3
• Once uploaded to S3 can choose to be notified trigger SNS
• Events can also be passed to the group CloudWatch monitoring logs
• Log files in S3 using SSE encrypted storage, you can define the life cycle of the log archive or delete
• API calls will generate a log in about 15 minutes
• Log files are released every 5 minutes
• Enabled by default, and record retention 90 days

Configuration

• It applies to all areas of CloudTrail
• Each Region using the same configuration and policy
• All the log will be transferred to the individual bucket designated S3
• This is the default configuration CloudTrail, is the recommended option
• It applies to a single region CloudTrail
• Processing each zone own log Trail
• Each log to their region S3 bucket

CloudTrail tracking

• By setting CloudTrail tracking, you can CloudTrail events to Amazon S3, Amazon CloudWatch Logs, Amazon CloudWatch Events. This allows you to use a variety of features to help archiving, analysis and response changed AWS resources.
• Will be applied to all areas of a track refers to create a track record in all areas of AWS account activity.
• You only need one API call or a few clicks of the mouse, you can in all regions of the sub-region to create and manage track. You will receive a record across all areas of your AWS account account activity carried out in an S3 bucket or CloudWatch Logs log group
• Global Trail
  • After a tracking applied to all regions, CloudTrail by copying the relevant tracking configuration will create a new track in all areas. CloudTrail the recording and processing of each area in the log file, and will contain all AWS regions account activity log files transferred to a S3 bucket and a CloudWatch Logs log group.
• Multiple Trail
• In an AWS area, you can create up to five track. Applied to all areas of the track will appear in each region, and count as a track each region.
• With multiple tracks, different interests of security administrators, software developers and IT auditors and other stakeholders can create and manage their own track.
• If you turn on multiple region Logging Global Service, Global Service logs will generate more than repeat, it is recommended that only one region to enable

CloudTrail processing library

• AWS CloudTrail processing library is a Java library that can help you easily build applications that read and process CloudTrail log files.
• CloudTrail processing library may provide the following function processing tasks, such as continually polls SQS queue, reading and parsing SQS message, download the log file stored in S3, and a fault tolerant sequence of the parse an event log file.

AWS Trusted Advisor

Outline

• Learn a lot of best practices, check the AWS environment, there is an opportunity for capital savings, availability, and performance, Address Security Vulnerabilities recommendations
• View the overall status of AWS resources through dashboards and budget savings
• Four categories of best practices
  • Cost Optimization
  • safety
  • Fault Tolerance
  • Performance Improvements
• Color coding:
  • Red - recommendations for action
  • ××× - recommended investigation
  • Green - no problems are detected
• Free checked items
  • Service restrictions - Check exceed service limit of 80%, based on a snapshot, about a 24-hour delay
  • Security group does not limit the port - the port to allow inspection of 0.0.0.0/0
  • IAM - Check whether IAM
  • Root account MFA - to check whether the root account enabled MFA

    Trusted Advisor features and functions

• Notification: free service, send an e-mail each week for updates AWS resources deployed
• Access Management: IAM use to control access to specific categories of inspection items or check
• AWS Support API: Programmatically retrieve and refresh Trust Advisor results
• Action links: follow the recommendations directly to the AWS Management Console through a hyperlink report
• Recent changes: changes in the recent follow-up examination on the console dashboard
• Exclusions: the custom is not checked unrelated projects
• 5 minutes Refresh: by clicking refresh all or automatically refresh every five minutes to check items

AWS Config

Outline

• AWS provides a list of resources, configure a fully managed service history and configuration change notifications
• Support compliance audits, security analysis, change tracking and troubleshooting resources
• By default, AWS Config configuration item is created as a resource area for each supported
• Every change will generate a configuration item change history
• Resources can be configured to check whether the changes violation of non-compliance and behavior marked by SNS to send notifications
• Support change management, continuous auditing and compliance, troubleshooting, and security event analysis
• AWS Config can be enabled based on regional
• AWS Config data can be aggregated across multiple accounts, the account may not span the predetermined rule

Config Rule

• Config rule representing the desired configuration of a resource, its assessment is based on configuration changes related resources AWS Config recorded. Rules for the allocation of resources assessment results can be viewed in the Control Panel. Use Config rules, you can assess the overall risk and compliance status from a configuration perspective view compliance trends over time, and to identify what configuration changes resulted from the resource compliance rules.
• Config rule does not directly affect the way end users of AWS. It was only in the allocation of resources to assess configuration changes have been completed by the AWS Config after recording. Config rule does not prevent users from making changes possible non-compliance.
• Config rule after rule evaluation resource configuration items (CI) is captured by AWS Config. It does not evaluate rules before pre resources or change the allocation of resources.
• By default, you can create up to 50 rules in the AWS Account
• Any rule can as a rule triggered by a change or established as a regular rule. Triggered by a change in the rules of AWS Config to perform the specified resource record any configuration changes. In addition, you must also specify one of the following:
  • Tags keys :( optional values): "Label key: value" to mean with the specified: any configuration "tab key value" resource records changes will trigger rule evaluation.
  • Resource type: to specify any configuration changes to any resource within a resource record types will trigger the rule evaluation.
  • Resource ID: any changes to the grounds resource type and resource ID specified resource record will trigger rule evaluation.
  • Regular rule is triggered at a specified frequency. Available frequency of 1 hour, 3 hours, 6 hours, 12 hours or 24 hours. Has all the resources on a regular basis rule applies to the rule of complete snapshot of the current configuration items (CI) of.

Config Items

• Configuration Item (CI) refers to a resource at a given point in time configuration. CI consists of five components:
• Basic information about the common resources of various resource types (such as Amazon Resource name, label),
• Resource specific configuration data (e.g. EC2 instance type),
• Mapping relationship other resources (e.g. EC2 :: Volume vol-3434df43 "attached to the instance" EC2 Instance i-3432ee3a)
• Associated with this state of AWS CloudTrail event ID,
• CI-related information to help you identify metadata (such as the CI version) and capture the time of the CI.
• AWS resource allocation Config detect and record changes to configuration changes caused by the state. If after another (for example, within a few minutes) multiple resource configuration changes, the Config will record only representative of this group to change the cumulative impact of the final configuration.
• With AWS Config, you can record configuration changes within the AWS EC2 instances in the account software, but also record the local environment in a virtual machine (VM) or server configuration changes. AWS Config configuration information record comprising an operating system update, the network configuration, installed applications, and the like.

AWS Config Integration with CloudTrail

• If you change the allocation of resources is the result of an API call, the AWS Config also recorded CloudTrail events and modifying the resource configuration API calls corresponding ID,
• While also recording the caller, calling the event logs and IP addresses, to facilitate troubleshooting.