Blog

  • AWS CloudWatch
    Image Credits: amazon.com
     
    Amazon Web Services has inbuilt functionality to good metrics on the console by default, like CPU, but it’s missing some key metrics that can cause bottlenecks like memory usage or disk space; these are essential to monitor to ensure instance uptime and health.
     
    The monitoring scripts are written by Amazon themselves, but aren’t included unless you set them up yourself, which isn’t always obvious. The scripts are available for different operating systems, however we will focus on Linux-based systems in this post. Amazon's documentation on CloudWatch Agent implementation, while comprehensive, is hard to find; hopefully this post will help you with your own instance monitoring. 
     
    Note: This post will be helpful for developers/devOps who would like to configure CloudWatch for disk and memory monitoring as the previous perl scripts provided by AWS is deprecated and not recommended and there is no other sources which mentions the steps clearly.
     
    AWS has how-to instructions on creating memory, disk and swap checks via Perl scripts located at https://aws.amazon.com/code/8720044071969977. These scripts are customised and need cron jobs to be configured to use designated intervals, every 5 minutes.
     
    Amazon introduced AWS CloudWatchAgent to automate this:
     
    Here is a few important list of the available metrics within the manual monitoring scripts:
     
    • Memory Utilization (%)
    • Memory Used (MB)
    • Memory Available (MB)
    • Swap Utilization (%)
    • Swap Used (MB)
    • Disk Space Utilization (%)
    • Disk Space Used (GB)
    • Disk Space Available (GB)
     
    Installation Process Overview
     
    The general flow of installing the CloudWatch agent is as follows:
     
    1. Create and Configure the IAM roles and users for the CloudWatch agent.
     
    2. If you are installing on an EC2 instance, attach an IAM role to the instance. Else, If you are installing on an on-premises server, you need to create an IAM user to enable the CloudWatch agent to write information to CloudWatch.
     
    3. Downloading the CloudWatch agent package, using either AWS Systems Manager Run Command or a public Amazon S3 download link.
     
    4. Modify the CloudWatch agent configuration files as per our requirement.
     
    5. Initiate the agent, using from the command line.
     
     
    How to Simplify Memory and Disk Utilization Monitoring in AWS
     
    1. Create an IAM Role and attach policy 'CloudWatchAgentServerPolicy'
    2. Attach the instance you want to monitor to the created role from EC2 instance console. 
    Select Instance --> Actions --> Instance Settings --> Attach/Replace IAM Role 
    3. SSH to your EC2 instance and follow the steps
     
    Downloading the CloudWatch Agent Package on an EC2 Instance
    Make a directory for downloading the AmazonCloudWatchAgent and then navigate to that directory
    Download the CloudWatch agent. 
     
    wget https://s3.amazonaws.com/amazoncloudwatch-agent/linux/amd64/latest/AmazonCloudWatchAgent.zip
    
     
    Unzip the package
     unzip AmazonCloudWatchAgent.zip
     
    Install the package
     sudo ./install.sh
     
    Open the CloudWatch Wizard and choose options
     sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard
     
    The config file generated will be at:
     /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json
     
    This file can be manually edited according to your requirements
     
    Start the CloudWatch
     sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a start
     
    CloudWatch Status can be checked at:
     sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status
    You'll see following status if CloudWatch is running
     
    {
      "status": "running",
      "starttime": "2018-05-03T11:03:30+0000",
      "version": "1.200442.0"
    }
    
    
     
     
    If required, CloudWatch monitoring can be stopped
     sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a stop
     
    CloudWatch logs will be generated at this path:
     /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log
     
    CloudWatch other paths are defined at this URL:
     
    Here is an example of standard configuration file to monitor cpu, disk, memory & swap
     
     
    
    {
    	"metrics": {
    		"append_dimensions": {
    			"AutoScalingGroupName": "${aws:AutoScalingGroupName}",
    			"ImageId": "${aws:ImageId}",
    			"InstanceId": "${aws:InstanceId}",
    			"InstanceType": "${aws:InstanceType}"
    		},
    		"metrics_collected": {
    			"cpu": {
    				"measurement": [
    					"cpu_usage_idle",
    					"cpu_usage_iowait",
    					"cpu_usage_user",
    					"cpu_usage_system"
    				],
    				"metrics_collection_interval": 60,
    				"totalcpu": false
    			},
    			"disk": {
    				"measurement": [
    					"used_percent",
    					"inodes_free"
    				],
    				"metrics_collection_interval": 60,
    				"resources": [
    					"*"
    				]
    			},
    			"diskio": {
    				"measurement": [
    					"io_time"
    				],
    				"metrics_collection_interval": 60,
    				"resources": [
    					"*"
    				]
    			},
    			"mem": {
    				"measurement": [
    					"mem_used_percent"
    				],
    				"metrics_collection_interval": 60
    			},
    			"swap": {
    				"measurement": [
    					"swap_used_percent"
    				],
    				"metrics_collection_interval": 60
    			}
    		}
    	}
    }
    
    
    

Tags: Web Apps