Install and Configure the Cloud Watch Logs Agent on a Running EC2 Linux Instance

 

 

 

What is Cloud Watch?

Cloud watch is a monitoring service to monitor your AWS resources,
as well as the applications that you run in AWS. So the cloud watch basically monitors performance. Performance monitoring and cloud watch can monitor things like compute. Inside the compute, it can monitor things like our
EC2 instances, Auto scaling groups, Elastic Load balances, and Route 53 health checks. Cloud watch can go through and not just monitor the EC2 instances it can monitor a whole bunch of other things as well. It can monitor storage and content delivery, our EBS volumes we can monitor how busy the volumes are we can monitor storage gateways and we can monitor cloud front using cloud watch as well.

 

Cloud watch Pricing 

It is Free 

  

 

 

 

Scenario-:

Install and Configure the CloudWatch Logs Agent on a Running EC2 Linux Instance

To do this you have to log in to your Amazone account

  

 

Before Enable the Cloudwatch we have to do a few things

1. We have to create a Policy

2. We have to create a Role

3. Attach the IAM role To the EC2 Instances

 

 

  Creating a Policy

 

 

 

To Create a Policy we have to go to the IAM

In IAM from the left side, panel select Policy tab to Create a new Policy 

Click Create Policy

In Policy Window Select JSON tab and add the below JASON Policy

 JASON

 

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents",
        "logs:DescribeLogStreams"
    ],
      "Resource": [
        "*"
    ]
  }
 ]
}

 

Click Revie Policy

 

In the Review policy window adds the Name and Description for the Policy. 

Note -: You can use any name that you like

Create Policy

 

After Creating the Policy you will be able to see the Policy that you have created

 

 

Creating a Role

 

 

In IAM from the left side, panel select Role tab to Create a new Role

 

Click Create Role

 

In the Creating Role  Window

Select AWS services  and select EC2

Click Permissions

 

In the Permissions Window Select the Policy That you have created Earlier

 

In here Permission will be granted from the policy

Click Tag

 

In the Tag Window, You can give a Tag for the Role in my case I gave as 

Key -: Name

Value(Optional) -: CloudWatchLogAgent

 

 Click Review

 

In the Review window, you can give a Name to your Role

I gave the same name as it was in the Tag

After Creating the role you will be able to see your role

Attaching an IAM Role to a Instances

 

 

 
Attach the IAM role and add the Details

IAM Role Select the Role That you have Created in the Role

Install and Configure the CloudWatch Logs Agent

 

 

To Install the Cloud watch agent we have to log in to our Instances

Use the Putty and your Public IP to log in to your Instances 

After logging to the Instances  use the Below Commands to Install  the Cloud Watch Agent

Fist Update your Instances

sudo yum update -y

Install the  AWS Logs

sudo yum install -y awslogs

Start the AWS Log Services

sudo systemctl start awslogsd

Check the Logservicese Status (It should be active (running) )

sudo systemctl status awslogsd.service

Enable the Logservicese at each boot

sudo systemctl enable awslogsd.service

 

Note -: By default, the /etc/awslogs/awscli.conf points to the us-east-1 region, if you are in a different region edit the awscli.conf and add the Region that you are in. You can check your Region in top Right Conner

 

 

Check the Cloud watch Logs

 

 

To Check the Log we have to go to the Cloud Watch

Services under Management & Governance select Cloud Watch

Click Logs on the left to view the logs

Here it will show the Path  that the Instance Log file is saved in

Click on the Path (/var/log/messages)

 When you click on the Path you will be able to see the log file

 Select the Log file and click it

 

This is the in datil of the Log file you can use filter events to Filter the Log 

For instance, you can usee Keyword Erro to filter all errors.

 

 

Real-world Example

Using Cloud watch we can monitor the Instances log and we can have a good idea about our instance such if there are errors in that we can easily motor those using cloud watch. Moreover using these we can monitor the security issues that we are facing such as if someone has tried to hack our Server we will be able to see how many times have they tried etc..

 

 

References -:  

 

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/QuickStartEC2Instance.html

https://aws.amazon.com/cloudwatch/pricing/

                                

Create Elastic Load Balancing and Auto Scaling Group

 

 

What is a Load Balancer? 

The load balancer is exactly what it sounds like It is a physical or virtual device that's designed to help you balance the load balance the network load across multiple web servers. if we've got a load balancer HTTP web server One, Two webserver Three, etc. It can balance the load across the three of them. You can also use it for applications it doesn't have to necessarily be Internet-facing load balances. but typically they are Internet-facing and primarily they're used to balance load across web servers.AWS has three different types of load balances

Load balancers Type

1. Application load balance (Intelligent Layer 7)

2. Network load balancer (Extreme Performances Layer 4 )

3. Classic load balancer (Basic Load balancers with Low Cost)

    

Image Sources

Create load balancer with Autos scaling group

To do this you have to log in to your Amazone account

 

 Creating an Image from the Existing Instances

 

 

First of all, we are going to create an Image from the existing Instance then they both will be identical.

To do that go to the Ec2 instances and select the Web Server 1 then got to Action and Select Create Image and give the detail as below then Select Crate Image

 

 Image name: Web Server AMI

 Image description: Lab AMI for Web Server

 

 

Creating a Loadbalancer

 

Select Loadbalancer form the left side panel then Click Create Loadbalance.

You will get a three options namely -: Application Load Balancer,  Network Load Balancer, Classic Load Balancer

For this  Lab, we will select Application Load Balancer and add the details  below

 

 Name: LabELB

 VPC: LabVPC

 Availability Zones: Add the Both availability subnets 

Click Configure Security Settings

 

In the Security Settings

Select the Existing  Security Group

From the list Select Only the Web SecurityGroup

 

 

Click Configure Routing

 

 

 

In the Routing add the Details as below

Name: LabGroup 

Keep all the setting as the default

 

Click Register Targets 

 

In the Register Targets, you don't want to select anything just keep this setting as the default  

 

Click Review and  select Create

 

 

 Configuring the Auto Scaling Group

 

 

Here we are going to create the Configurations for the Auto Scaling this going to be the other scaling instances

Form the left side of the panel select Launch Configurations. 

 

Select the Create Launch Configurations. Under AMI  select My AMIs Then You will be able to see the Web Server AMI that we have created in the First step of this Lab, Select it 

 

 

 

 In Instance Type select

 

Instance Type as- t2.micro (It will be the default selected Instances)

Note: If you have launched the lab in the us-east-2 Region, select the  t3.micro instance type

 

  Click Configure details

 

 

 

Add the details as Below for the Configure details  Select Add Storage 

 

Name: LabConfig

Monitoring: Put the Tick for Enable CloudWatch detailed monitoring

 

In the Add, Storage keep the storage settings as the default

 

Click Configure Security Group

 

In the Security Settings

 

Select the Existing  Security Group

from the list Select Only the Web SecurityGroup

 

Click  Review 

 

After Clicking the Review  In the next window  

Click Create Launch Configuration

 

 

 

Then It will ask for the Key Pair select

Existing key pair from the Dropdown List

Give the key pair as  vockey

Select Create launch configuration

 

 

 

When the instance is created You will geta a message saying

 

Create an Auto Scaling group using this launch configuration  Select this 

Note-: Don't Select Close Button   

 

 

In the Configure Auto Scaling Group Details add the Details as below

Group name: Lab Auto Scaling Group

Group size: 2

Network: LabVPC

Subnets: Private Subnet 1 (10.0.1.0/24) Private Subnet 2 (10.0.3.0/24) This will help EC2 to work in Both AZ

 

 In the same page Under Advanced Details

 

Add the details as below 

 

Load Balancing: Put the Tick for Select Receive traffic from one or more load balancers

 Target Groups: LabGroup

 Monitoring: Put the tick for Select Enable CloudWatch detailed monitoring

 

Click Configure scaling policies

 

In the scaling policies, Select Use scaling policies to adjust the capacity of this group. and add Scale between  2 and 6

 

 In Scale, Group Size Add the details as below 

Metric type: Average CPU Utilization

Target value: 60

 

Click Configure Notifications

 

In the next window Configure Notifications

Keep the default Notification setting as it is

 

Click Configure Tags

 

 

 In the Tag add the details as below

 

 Key: Name

 Value: Lab Instance

Click  Review

 

Here You will be able to see the datils of your Autoscale Group 

 

Click Create Auto Scaling group

 

 

At the First time When you Click the Create Auto Scaling group, You will get this message 

 

Click Retry Failed Tasks

 

 

 After CLicking the Retry Failed Tasks  it Should Successful then Click Close

 

 

Verify that Load Balancing is Working

 

 

Select Target group form the left side panel, you will be able to see the Group name call LabGroup Click that one and after clicking that you will get this window. In this window select the Targets tab then you will be able to see the health of the targets The Two Targets Status Should be  Healthy.

 

 Note -: In the Instances, you should see two new instances named Lab Instance

 

Select Loadbalancer form the left side panel and select LabELB in the under the description menu (bottom of the window)  get the DNS name of the  Loadbalancer open a new tab in your Web Browser and paste the copied link then you will be able to see a web page as above.

Note-: If You Noticed carefully When you Click the AWS Log, The Availability Zon will change between (us-east-1a and us-east-1b) when the page is refreshed. This means the load is balanced between two AZ.

 

 

 

 Test Auto Scaling

 

 

To test  the Auto Scaling We are going to use Cloud Watch Services 

In the services  Under Management & Governance Select CloudWatch

From the left panel, click Alarms Then You Should be able to see the details as above

 

When The page is Loaded to the page You will be able to see the Alarms has been triggered

 

 

 

 

In this Alarm, it shows that  CPUUtilization>60 for 3 datapoints within 3 minutes

This means that the Intncase is overload and now it will autoscale and it will add more Instances.

 

 

In the Instances, you will be able to see the newly added Instance for Autoscaling 

 

 

 

What are the benefits and real-world example

If we think that we have a ticketing Web site and when the season comes it's peaking. So, for those days only, we need to balance our load on the Web. For this kind of situation, we can create load balancers in Multiple AZ when the load hits the web site it will balance the load across the AZ. Moreover, using Auto Scaling it will create the Instances it will help the webserver to configure resources automatically depending on requirements. Using these two services we can achieve some great benefits such as Increased Scalability, Redundancy, Reduced Downtime, Increased Performance are a few of them.

 

 

 

 

 

 

 

  References-:  

https://aws.amazon.com/elasticloadbalancing/

 

https://aws.amazon.com/elasticloadbalancing/

 

 

 

Build DB Server and Interact With DB Using an App

What IS DB?

The relational databases are what most of us or all used to. They've been around since the 1970s.AWS has come in six different Variations relational database services, it has two key features. We've got multi-AZ which is for disaster recovery and then we've got read replicas which are for performance.multi-AZ Disaster Recovery read replicas performance

 

These are the six different relational databases on AWS

• Microsoft sequels server

• Oracle

• my SQL Server

• PostgreSQL

• Aurora

• MariaDB

What are the Benefits

• Easy to administer

• Highly scalable

• Available and durable

• Fast

• Secure

• Inexpensive

RDS Have Two Key Features

Multi-AZ - Disaster Recovery

Read Replicas - For Pereformenceses

 

Multi-AZ

Multi-AZ - Disaster Recovery

In the above example. I've got my example D.B. (There are Two AZ's)
So we've got our Easywto instance up the top and it connects into our databases using a connection and there is the address US East1.rds.amazonaws.com, Amazon host this DNS record and they point it at a database a primary database as they've pointed at it's in its internal IP address. For some reason we lost our Primary DB Amazon would detect that and they'd automatically update the DNS to point to the secondary. So you don't need to go in and change IP addresses or anything like that All you have to do is pointing out to this DNS address an Amazon will go in and update that DNS address with the new IP address. So failover is automatic with Multi-AZ

Read Replicas

Read Replicas - For Pereformenceses

In the above example. I've got my example D.B. (There are Two AZ's) We've got our Easytwo instances and they're pointing off to our primary instance using this connection .and what happens is that every time you add Data to that database that wright is going to be replicated to another database. It's a good perfect copy of the other database but it's a replica. Now if we lose our primary database for whatever reason yet, there's no automatic failover from one to the other. What you would have to do is you'd have to go in and have to create a new URL and then you'd have to update your Easytwo instances to point to the Read Replica.

This is the Scenario

After configuring everything it should be like this

In this scenario, we are going to create a Database with the subnet groups, Backup DB that is in another AZ. Using a Web interface we are going to update the DB.

To do this you have to log in to your Amazone account

 

 

Creating the VPC Security group

To do this we have to Create a VPC Security group

In the VPC Select Create Security group

In the VPC Select the VPC that you want to add to the Security Group

 

After Creating the Security Group Add an Inbound Rule the VPC to do that select the VPC Menu from below menu select Inbound rules and select Edit Inbound Rules and add the Below details

Type -: MYSQL/AURORA

Select the Web Security Group

Here what we are doing is We are allowing (Talke) to the  Web Security group into the DB Security Group

Creating the DB Subnet Groups

Now We have to create the Database to do that from the services under data base select RDS

Then Select DB Subnet Group this

To create the Subnet Group Add the details below

Name -: DBSubnetGroup ( Do not keep space you won't be able to create it )

Description -: DB Subnet Group

Under Add subnets

Availability zones   Add the Two Availability zones  (That suite for your Region)

Subnets Add the 2 Different Subnets 

Then Select Create 

 

 

Creating the DB

After Creating the DB Subnet Group Now we have to create the DB To do That in the same window Top left conner Select Databases

In this Window, We are Going to Select the DB Platform and we have select it as MYSQL

 

In the settings add the Details below to Create the DB

We will select the Production Template - It will give us the High availability, Fast Consistency performance, and High performing disk. This will give you the Multi-AZ

DB Instance identifier -: lab-db (RDS Instances name )

Master Username -: master

Master Password -: password (you can give any password you like )

DB Instance Class -: Burstable classes (includes t Classes)

from the drop-down menu select-: db.t3.micro

In Storage

Storage Type General purpose (SSD)

Allocated Space 20 GB

Keep these setting as the default 

Enable Storage Autosacaling- This means that you can add storage as run out of space

 We can Specify the maximum storage that we are going to use 

 

Under the Connectivity

Select The VPC-: LAB VPC

Subnet Group -: dbsubnetgroup (The subnet we have created earlier )

VPC Security Group Select Chose existing Group

From the list select DB Security Group (Do not select two)

Initial Database Name -:lab - If do not configure an Initial Database name Amazon RDS does not create the Database name for us and we need to use PHPMyAdmin or some way to Provisig or using sequel queries to create the database.

Untick  Enable automatic backup (Not Recommended at all we should Keep this tick for quick deploy we will untick it now )

Untick Enable Enhanced monitoring

Here you will be able to see the cost for your DB with details 

Keep all the setting as default and select Create

DB will be created and if you Click the lab-db you will be able to see the status of the DB

It will take a few minutes and there will be 3 stages Creating Modifying and Available

When the staus change to available it will be ready to use 

Copy the Endpoint Link the Highlighted one This will help you to connect to the RDS from the Web interface 

 

 

 

Login to the DB using web Interface 

Get the Incance IP address  Open a new tab and  enter the Instance IP address

 

After You got the web interface Select RDS in the web page from top  Next to the AWS logo

then enter the details 

Endpoint the Link that you have copied from the database

Database name -: lab

Username-: master

Password -: Password

After adding all the details Click Submit

It will take 10 seconds  then You will get the above page as DB you will be able to add details for this DB

Click Add contact to add the details

After adding the Contact details 

 

 

 

Problems

I have seen one Issue with this database that is we can add Characters to the Number field as you can see for the Phone number I was able to add Characters.  In the Email also it does not Check, Users can add anything.

 

How would It help us in Real-world AWS RDS is the easy way to deploying a DB cause we do not wont to worry about the Instances cause Amazone do the Patching and updating. We have to worry about our DB only. It also supports most database Platforms.

-------Important---------

Amazone RDS is not Serverless

Aurora Serverless is Serverless services

References -: 

https://aws.amazon.com/rds/

 

https://aws.amazon.com/rds/features/multi-az/

 

https://aws.amazon.com/rds/features/read-replicas/