CloudWatch

General Info

Use cases	Monitoring service for AWS resources/applications: Collects monitoring and operational data in the form of logs/metrics/events Provides a unified view of AWS resources, applications and services Can set high resolution alarms, visualize logs and metrics side by side, take automated actions, troubleshoot issues, and discover insights to optimize applications
Enables	Resource utilization, operational performance monitoring Provides: disk read operations, CPU usage, and inbound network traffic Does not provide memory usage Real-time monitoring Log aggregation and basic analysis
Limits	5,000 alarms for each AWS account Metrics data retained for 2 weeks by default

CloudWatch Logs

• Allows to aggregate and monitor logs from applications and systems
• Sources
  • Pushed from some AWS services (including CloudTrail)
  • Pushed from your apps/systems
  • Metrics from log entry matches

Elements

Type	Description
Log Event	A record representing an event Contains Timestamp (date and time when the event occurred) Raw message (specific data thr record is reporting)
Log Stream	A sequence of log events that share the same source
Log Group	A collection of log streams that share the same retention, monitoring, and access control settings
Subscription Filters	Define a filter pattern that matches events in a particular log group, send them to Kinesis Data Firehose stream, Kinesis stream, or a Lambda function

Log Group Configurations

Configuration	Description
Encryption	Can Use KMS, CMK (Customer Key)
Retention	By default, there is no retention limit (does not use S3), unless you set a retention period Can configure the retention period in days (1 - 3,653) Can export Log Groups (in a particular time range) to S3 (not real time, up to 12h)

Integrations

• CloudWatch Agent
  • Can be installed on a host (e.g. via SSM) to push logs to CloudWatch Logs
• Cross-Account
  • Can receive events from other accounts by creating a destination in CloudWatch, which references a receiving Kinesis stream
  • The destination has a resource-based policy that controls which accounts can write to the destination
  • CloudWatch Logs on the sender side can then stream to the other account
• CloudWatch Logs Insights
  • Provides a query syntax to apply to one or more (up to 20) log groups

  • Aware of the system fields that are automatically generated when a log event arrives to CloudWatch Logs:

    Field	Description
    @message	raw log event
    @timestamp	when the event occurred
    @ingestionTime	when the event was ingested
    @logStream	the name of the log stream
    @log	the unique ID of the log group

Metrics

• CloudWatch as a metric repository service: records information in a time sequence by storing a number (metric) for each period of time

• Ingestion

  • At the time of ingesting data into CloudWatch, the sources define:

    1. A namespace
    2. A metric name
    3. Metric dimensions to group the information (optionals)

    4. Resolution

       Resolution	Granularity	Notes
       Standard	1m	Default
       High	1s	Only available for custom metric

  • Retains the metrics for up to 15 months

  • CloudWatch Application Insights
    • Collect information about specific apps via an SSM Agent
    • Compatible with .NET and SQL Server apps
  • Alarms:
    • Defined thresholds for a specific metric
    • Once a metric crosses the threshold for a custom-defined number of times, the alarm is triggered
    • Can configure actions to execute and notifications to send when an alarm is triggered

CloudWatch Monitoring

Characteristics

• Provides monitoring of performance metrics (real time)

• Types

  Type	Description
  BASIC	Sends data points to CloudWatch every 5mins For a limited number of metrics No charge
  DETAILED	Every 1min Allows data aggregation (across AZ within a region) Additional charge

• Metrics

  • Hypervisor visible metrics (CPU) (NO MEMORY)
  • default = CPU util/network util
  • custom = disk space/RAM utilization
• Alarms = if CPU > 80% for 5 mins —> alarm
• Notifications = SNS
• Can support on-premise services (doesn't need to be an AWS service)

CloudWatch Events

Characteristics

• Provides a near real-time stream of events within your AWS account which can be used to trigger actions (such as a Lambda function) to perform a task
• Near real-time stream of system events (event-driven security)

Events	AWS CloudTrail (API calls) AWS resources state change Custom events (code) Scheduled
Rules	Match incoming events and route them to 1+ targets Rules that trigger from either event patterns or a schedule Rules send JSON to one or more targets
Targets	AWS lambda functions, SNS topics, SQS queues, Kinesis Streams Lambda function can kick in a delete the newly created EC2 instance
Access control	Use IAM policies to restrict access to CloudWatch and the Actions they can perform However, data is decoupled from its source, therefore you are not able to restrict access by the originating resource

EventBridge

Use Cases

• Delivers a stream of events representing changes to resources (preferred option over CloudWatch Events for managing events from AWS resources)
• Provides services and applications with an easy way to react without the need to configure the response actions inside themselves
• Provides abstracted event buses and adds capabilities to connect sources of events with action executors

Rule Components

Component	Description
Event Buses	Default Automatically receives events from AWS resources Unique bus per region and per account You don't create it and you cannot delete it because it is part of the EventBridge infrastructure Partners Custom
Event Pattern	An expression to match the events of interest
Target	A service that will be invoked The target must be in the same region as the rule For the same rule, you can configure several targets For each target you can define an event transformation that describes the JSON object that will be delivered to the target

Custom EventEvent pattern for custom eventPattern to detect IAM events delivered by CloudTrail

{
    "Entries": [
        {
        "Detail": {
            "custom-app-id": "id-2131580",
            "role": "accounting"
        },
        "DetailType": "Custom App Sample Event",
        "EventBusName": "my-event-bus",
        "Resources": [
            "chile:santiago:west-1:sensor/12345"
        ],
        "Source": "custom.app.001",
        "Time": 1589627829
        }
    ]
}

{
    "account": [
        "123456789012"
    ],
    "source": [
        "mycustom.app.001"
    ],
    "detail-type": [
        "My Custom App Sample Event"
    ],
    "detail": {
        "mycustom-app-id": [
        "id-2131580"
        ]
    }
}

{
    "source": [
        "aws.iam"
    ],
    "detail-type": [
        "AWS API Call via CloudTrail"
    ],
    "detail": {
        "eventSource": [
        "iam.amazonaws.com"
        ]
    }
}