AWS EC2 Capacity Reservation

Use EC2 Capacity Reservation where your require future EC2 capacity based on a specified set of requirements.

Unlike other EC2 Instance Reservations there is no one to three year reservation commitment, you will be billed as soon as the instance starts

When reserving EC2 capacity you must specify the Availability Zone for the EC2 instances (Limited to a single Availability Zone), the number of EC2 instances you require and any instance attributes such as the Instance Type, Operating System and Tenancy Type (e.g. Run as a shared hardware instance).

AWS EC2 Nitro

EC2 Nitro is the next generation of EC2 instances leveraging new virtualisation technology and will allow for much better performance, specifically:

  • Better networking performance, IPv6 IP support, High Performance Computing
  • Nitro Instances provide better security and much higher EBS IOPS (IO Operations Per Second) support. Current non Nitro EBS volumes can only support a maximum of 32 000 IOPS whereas Nitro can support up to 64 000 IOPS

Hibernating EC2 Instances

Aside for stopping and terminating EC2 instances there is a third option which allows us to hibernate an EC2 instance.

Currently when we stop an EC2 instance any data on our EBS volume is saved until the EC2 instance is started again.

If we terminate an EC2 instance any EBS information which is at a root volume level or has been designated to be destroyed will be lost. Note, any EBS volume which is attached a secondary volume to an EC2 instance and is not setup to have it’s data destroyed on termination of the instance will not lost.

So, what exactly is EC2 Hibernate? Think of putting your EC2 instance into a sleep mode. When an EC2 instance is placed into Hibernate mode the state of the instance is written to disk e.g. The Operating System is not actually stopped which means that starting the instance is much faster.

For security reasons the instance state which has been written to the EBS volume is encrypted

The following are some important notes surrounding hibernation and EC2 instances:

  • Not all EC2 instance families are supported and there is a 150 GB limit on the RAM size of the EC2 instance.
  • Bare Metal EC2 instances are not supported and the root volume of the EC2 instance must be EBS, not an instance store, large enough to be able to save the state of the EC2 instance and also encrypted
  • Hibernation is only available for On-Demand and Reserved EC2 instances (Not Spot Instances) and there is a sixty day maximum limit on the amount of time that an EC2 instance can be hibernated.

AWS Elastic Network Interfaces

Otherwise known as ENI are a virtual network card and are used to give EC2 instances network access and are able to be used outside of EC2 instances as well.

An AWS ENI is bound to single Availability Zone.

It is feasible for an EC2 to have one or more ENI associated with it

Each AWS ENI is associated with a primary private IPv4 IP addresses and one or more secondary IPv4 IP Addresses.
An AWS ENI is able to have a single Elastic IPv4 IP Address for each private IPv4 IP address as well as a single public IPv4 IP address

An AWS ENI can be associated with one or more Security Groups as well as a MAC Address.

It is possible to create an independent AWS ENI and attach the ENI on an ad-hoc basis to EC2 instances for failover

EC2 Placement Groups

When creating EC2 instances it may be preferrable to dicate where you would like your EC2 instances to be placed in terms of Availability Zones. This strategy can be defined via what is known as Placement Groups

When creating a Placement Group you can choose one of the following strategies:

  • Spread – Indicates that the instances will be spread across different hardware across different Availability Zones (There is a limit of seven instances per Availability Zone) and this is a good use case for critical applications. Use case is where any EC2 instance failure does not impact another EC2 instance.
  • Cluster – Places the instances into a single rack in the same Availability Zone which will result in lower latency but with higher risk as if the Availability Zone or rack goes down then so will the entire set of EC2 instances.
  • Partition – Similar to the Spread Placement Group but the instances are spread across many different partitions. Instances are spread across partitions in multiple Availability Zones. Each partition represents a rack in AWS and therefore your instances are distributed across many hardware racks in AWS and therefore are safe from a rack failure from one another. There is a limit of seven partitions per Availability Zone and you are able to have hundreds of EC2 instances. Instances in a partition do not share racks with instances in any other partition meaning that each partition is isolated from failure.

Use case for partition is where an application is partition aware such as Big Data applications such as Cassandra, HDFS, Kafka etc.

AWS Elastic IP

AWS dynamically changes the public IP address of an EC2 Instance when the instance is restarted.

As such in order to have the same IP address when an EC2 Instance is restarted it is necessary to have an AWS Elastic IP Address. An Elastic IP address is a public IPv4 address which can be attached to a single EC2 instance at any one time and is associated with an AWS account until it is deleted.

One use case for an Elastic IP address is to easily remap to a different EC2 instance if one EC2 instance should fail. However due to there being a 5 Elastic IP limit for your AWS account this is not a common pattern.

Best practice is to avoid using Elastic IP Addresses all together and instead register a DNS name (See Route 53).

Alternatively use an AWS Load Balancer and do not use a public IP address at all.

Which EC2 Instance should you use?

EC2 On Demand Instance

The most expensive instance launch type but only pay for what you use with no upfront payment and no commitment.
Available with Mac OS (Billed per hour), Linux and Windows operating systems (Billed per second of use after the first minute)

Use case is for short-term use where the instance cannot be interrupted and where it is impossible to predict how the application will behave

EC2 Reserved Instance

An EC2 Reserved Instance is an instance where you are prepared to commit to leasing the instance for a specific period of time (Between one and three years).

The longer you commit to having the instance the more discount you will receive. As an example you can save up to 75% off the price of the same On Demand instance.

In addition if you pay up front for the instance you are able to make further savings. As a further example if you commit to a three year lease of the EC2 instance and pay for all three years up front then you will make substantial savings.

The use case for Reserved instances is for applications which will stay in the same state e.g. Databases

There are two other options when it comes to Reserved Instances:

Convertible Reserved Instance where you are able to change the type of EC2 instance. Choosing this instance type will not be as cost effective and will bring savings of up to 54% over and On Demand instance

Scheduled Reserved Instance (Deprecated). An reserved instance which will only be used during a specific timeframe during day / week / month

EC2 Spot Instances

The most cost effective instance with savings of up to 90% over On Demand Instances. However, EC2 Spot Instances can be terminated at any time if another AWS user outbids the price you are willing to pay.

Therefore EC2 Spot Instances are not good for any critical work loads (e.g. Databases) and should only be used for workloads that are resilient to failure e.g. Media processing, Batch Jobs, Distributed Workloads

When bidding for a Spot Instance you bid the max spot price you are willing to pay and you will have a spot instance while you are the maximum bidder.

The price per hour is based on the offer and capacity.

If and when your maximum price is outbid then you have a two minute grace period and you can chose to either stop or terminate your instance.

Where there is a requirement that a spot instance must not be terminated you are able to purchase a Spot Block which is an EC2 instance which can be reserved for a period of between one and six hours without being interrupted.

When requesting an EC2 Spot Instance the following must be specified:

  • The maximum price you are willing to pay for the Spot Instance
  • The number of instances you will require
  • The specification of the instance
  • The type of request e.g. Persistant or one-time
  • The timeframe that you require the instance

One-time Instance Request: As soon as the instance request is fulfilled the instance request will be flagged as being fulfilled and will then go away

Persistent Instance Request: The process will keep instantiating instances for the request number of instances within the given timeframe.

e.g. If you specify that you want four requests and an instance is terminated or stopped then the request will be fulfilled based on the spot price.

A Spot Instance can only be cancelled where the Instance falls into one of the follwing states: Open, Active or Disabled. Additionally, even if a Spot Instance request is cancelled then it is up to the end user to cancel any running Spot Intances which have been instantiaed by the request

Therefore if you wish to cancel a Spot Instance request you must first cancel the Spot Instance Request and then terminate any running EC2 instances which were created by the request

EC2 Dedicated Host

An EC2 Dedicated Host is where you rent a physical server within the AWS Data Center and the server is dedicated to your use only and must be reserved for a three year period.

An EC2 Dedicated Host is more expensive than other EC2 instances.

The use case for a dedicated server is such where compliance requirements must be met.
Additionally, running a dedicated server may be more cost effective as you are able to install your own server bound licences (e.g. You may have an existing licence for Windows Server and MSSQL).

EC2 Dedicated Instances

An EC2 Dedicated Instance is where you rent an instance which is running on hardware which is dedicated to you and you will share the hardware with other instances in the same AWS account.

In addition you have no control over the instance placement and have no access to the underlying hardware of the instance – With a EC2 Dedicated Host you have full control of the server whereas an EC2 Dedicated Instance is more of a Software version.

Types of EC2 Instances

The following are the currently available instance types within AWS:

  • General Purpose
  • Compute Optimised
  • Memory Optimised
  • Accelerated Computing
  • Storage Optimised

Each instance type has multiple options and have the following naming convention:

m2.2xlarge

where m is the instance class
5 is the generation
2xlarge represents the size within the instance class

General Purpose Instance Types are geared towards workloads such as code repositories or web servers and have a good balance between memory, computing power and networking.

Compute Optimised Instance Types are geared towards workloads which require processor intensive workloads such as media processing, batch processing, machine learning and gaming servers

Memory Optimised Instance Types are geared towards workloads which require a large memory resident dataset – think in memory BI databases, high performance databases, distributed caching systems

Storage Optimised Instance Types are geared towards workloads where storage is a factor such as databases, distributed file systems, OLTP systems and data warehousing applications

Amazon EC2

Amazon EC2 stands for Elastic Compute Cloud and is not a single service but should be thought of as many services running as one; think of EC2 as Infrastructure as a Service

Amazon EC2 mainly consists of the following:

  • Renting virtual machines (EC2)
  • Storing data on virtual drives (EBS)
  • Distributing load across machines (ELB – Elastic Load Balancer)
  • Scaling the services by leveraging an auto-scaling group (ASG – Auto-Scaling Group)Understanding EC2 is fundamental to knowing how the cloud works

When creating an EC2 instance a user will need to make the following decisions:

  • What operating system will be installed on the instance – The current available options in AWS are: Linux, Mac OS and Windows
  • How much processing power will be required (CPU / Cores)
  • The amount of RAM the instance will require
  • Storage space required for the instance – Network attached storage such as EFS and EBS or actual hardware (EC2 Instance Store)
  • Networking such as the type and speed of the network card and IP addresses
  • Security group and Firewall rules
  • Any EC2 User Data scripts which will be execute the first time that the instance is powered on. Use the EC2 User Data Script to install software, install updates, download information from the Internet etc.

Note that the EC2 User Script runs as the Root User

IAM Best Practices

Never use the root account for any other purpose other than setting up other less privileged AWS accounts
One AWS user is the same as one physical user
Users can be assigned to groups and then permissions can then be assigned to those groups
Always enforce a strong password policy
Wherever possible, use Multi-Factor-Authentication
Whenever an AWS Service requires permissions to run as a user then create and assign the service a Role
When leveraging the CLI or SDK (Or another programmatic access) generate Access Keys for the application
Use the IAM Security Tools available within AWS such as IAM Access Advisor to audit user permissions on a regular basis
Never share IAM users and access keys – Each user should have their own Access Key