In this article, we have developed a Python script to simplify the process of retrieving AWS CloudWatch metrics for Elastic Block Store (EBS) volumes. The script takes input from a CSV file, containing metrics such as metric-name, VolumeId, START_TIME, and END_TIME, and uses the boto3 library to interact with AWS services.

By using this script, users can avoid manually executing individual AWS CLI commands for each metric and volume, making the process more efficient and less error-prone. The script iterates through the CSV file, calls AWS CloudWatch using boto3, and collects the required metric statistics, such as the Average value, for each metric and volume within the specified time range.

The output is then written back to a CSV file with the columns metric-name, VolumeId, Timestamp, and Average. This organized output allows users to easily analyze and further process the data for their specific use cases.

Users can customize the input CSV file with desired metrics and volumes, making it adaptable to various AWS environments and monitoring requirements.

SAMPLE - input.csv
metric-name,VolumeId,START_TIME,END_TIME
VolumeReadOps,vol-12345,2023-07-01T00:00:00,2023-07-02T00:00:00
VolumeWriteOps,vol-67890,2023-07-01T00:00:00,2023-07-02T00:00:00
BurstBalance,vol-54321,2023-07-01T00:00:00,2023-07-02T00:00:00
VolumeBytesRead,vol-98765,2023-07-01T00:00:00,2023-07-02T00:00:00
VolumeBytesWrite,vol-24680,2023-07-01T00:00:00,2023-07-02T00:00:00

CODE - sap_get_metric_statistics.py
import csv
import boto3

# Function to get CloudWatch metric statistics
def get_metric_statistics(metric_name, volume_id, start_time, end_time):
    cloudwatch = boto3.client('cloudwatch')
    response = cloudwatch.get_metric_statistics(
        Namespace='AWS/EBS',
        MetricName=metric_name,
        Dimensions=[
            {
                'Name': 'VolumeId',
                'Value': volume_id
            },
        ],
        StartTime=start_time,
        EndTime=end_time,
        Period=300,
        Statistics=['Average']
    )
    return response['Datapoints']

# Main function
def main():
    input_file = 'input.csv'
    output_file = 'output.csv'

    with open(input_file, 'r') as csvfile:
        csvreader = csv.DictReader(csvfile)
        next(csvreader)  # Skip the header row
        data = list(csvreader)

    with open(output_file, 'w', newline='') as file:
        csvwriter = csv.writer(file)
        csvwriter.writerow(['metric-name', 'VolumeId', 'Timestamp', 'Average'])

        for entry in data:
            metric_name = entry['metric-name']
            volume_id = entry['VolumeId']
            start_time = entry['START_TIME']
            end_time = entry['END_TIME']

            datapoints = get_metric_statistics(metric_name, volume_id, start_time, end_time)
            for datapoint in datapoints:
                csvwriter.writerow([metric_name, volume_id, datapoint['Timestamp'], datapoint['Average']])

if __name__ == "__main__":
    main()

SAMPLE - output.csv

metric-name,VolumeId,Timestamp,Average
VolumeReadOps,volume-1,2023-07-20 10:00:00,120.0
VolumeReadOps,volume-1,2023-07-20 10:05:00,130.0
VolumeReadOps,volume-1,2023-07-20 10:10:00,115.0
VolumeWriteOps,volume-1,2023-07-20 10:00:00,50.0
VolumeWriteOps,volume-1,2023-07-20 10:05:00,60.0
VolumeWriteOps,volume-1,2023-07-20 10:10:00,55.0
BurstBalance,volume-1,2023-07-20 10:00:00,75.0
BurstBalance,volume-1,2023-07-20 10:05:00,80.0
BurstBalance,volume-1,2023-07-20 10:10:00,70.0
VolumeBytesRead,volume-1,2023-07-20 10:00:00,2000.0
VolumeBytesRead,volume-1,2023-07-20 10:05:00,2200.0
VolumeBytesRead,volume-1,2023-07-20 10:10:00,1900.0
VolumeBytesWrite,volume-1,2023-07-20 10:00:00,1500.0
VolumeBytesWrite,volume-1,2023-07-20 10:05:00,1700.0
VolumeBytesWrite,volume-1,2023-07-20 10:10:00,1400.0