Amazon Elastic Block Store (EBS) is a service that provides storage for Amazon EC2 instances. EBS is what every volume on your EC2 instances runs on. When you rely on a service to run important infrastructure it’s wise to ensure that data is backed up. In this blog post, learn how to manage and automate AWS EC2 snapshots using the PowerShell scripting language.
Each EBS volume attached to an EC2 instance can be backed up via an EBS snapshot. Each snapshot can be created via one of two ways; a full or incremental snapshot. Well, technically both but..just go with me on this one.
When the first snapshot backup is created for an EC2 instance, it creates a backup of the entire volume. When subsequent snapshots are created, only data at the block-level storage level that have changed since the last snapshot are saved. However, unlike typical incremental point-in-time backups, since snapshots can be chained, when a snapshot is restored all of the data for that volume is restored making it similar to a full backup.
Snapshots can occur asynchronously meaning that snapshots can be in the creation process in parallel. When started, a snapshot enters a pending phase until all of the necessary blocks are copied to Amazon S3 where all EBS snapshots are stored.
Using PowerShell
There are a few different ways AWS allows you to manage EBS snapshots. You can use the AWS Management Console, the AWS CLI, PowerShell or the APIs directly if you’d like. In this article, we’re going to be using PowerShell since the AWSPowerShell module has great support for managing EBS snapshots with PowerShell commands.
Prerequisites
Before we get too far, there will be a few prerequisites I’ll be assuming you have to get the most from this tutorial. You will need:
• An AWS account
• An EC2 instance with an attached volume
• Authenticated with the root user or an IAM user with the CreateSnapshot permission (arn:aws:ec2:region::snapshot/*)
Once you’ve got all of these prereqs in order, we’re ready to go!
Finding Existing Snapshots
One of the first tasks we’ll do is to discover if we have any existing snapshots out there already. To figure that out, we use the Get-EC2Snapshot command. This is the command that will search for any snapshots that have already been created and will output them to the PowerShell console.
PS C:\> Get-EC2Snapshot
DataEncryptionKeyId
:
Description : Business/Industry Summary
(Windows)
Encrypted : False
KmsKeyId :
OwnerAlias : amazon
OwnerId : 947081328633
Progress : 100%
SnapshotId : snap-8af818e3
StartTime : 11/19/2008 6:15:17 AM
State : completed
StateMessage :
Tags : {}
VolumeId : vol-e1ac4888
VolumeSize : 15
<SNIP>
You will notice that the snapshots that come up though are not ones for your EC2 instance. These are all snapshots that have been shared with you and are available for you to restore to your EC2 instances. To discover all of the snapshots that have come from your EC2 instances, use the OwnerId parameter with the self value.
PS> Get-EC2Snapshot -OwnerId self
Creating a New EBS Snapshot
Assuming that you already have an EC2 instance created, you can now create a snapshot. You can create new snapshots using the New-EC2Snapshot command but first, you’ll need to gather up the volume ID for each volume you’d like to snapshot. To do that, we first need to find the EC2 instance ID where the volume is attached to.
PS> (Get-EC2Instance).instances
PS>
$instanceId = (Get-EC2Instance).instances.InstanceId
Once you know the EC2 instance ID, you’ll need to use that to find the volumes. To find all of the volumes attached to the EC2 instance, you’ll use the Get-EC2Volume command and filter the output to only the volumes with an attached instance ID of the one we’re looking for.
PS> $volumes = Get-EC2Volume | Where-Object {
$_.attachments.InstanceId -eq $instanceId }
Attachments :
{i-083007f62ff750d7a}
AvailabilityZone
: us-east-1b
CreateTime : 3/1/2019 7:06:57
AM
Encrypted : False
Iops :
100
KmsKeyId :
Size : 8
SnapshotId : snap-0ff5b79fdf9b021e8
State : in-use
Tags : {}
VolumeId : vol-0bf53c62534f99eee
VolumeType : gp2
Now that we have the volumes, AWS recommends shutting down the instance or at least dismounting the volume. We’ll go ahead and shut down the instance.
PS> Stop-EC2Instance -InstanceId $instanceId
We can now loop over each one and pass that volume’s ID to the VolumeId parameter on the New-EC2Snapshot command. Once that happens, you’ll then notice the snapshot will go into a state of pending.
foreach ($volume in $volumes) {
New-EC2Snapshot
-VolumeId $volume.VolumeId
}
DataEncryptionKeyId
:
Description :
Encrypted : False
KmsKeyId :
OwnerAlias :
OwnerId : 013223035658
Progress :
SnapshotId : snap-027a3550dde80eb3b
StartTime : 3/1/2019 7:19:13 AM
State : pending
StateMessage :
Tags : {}
VolumeId : vol-0bf53c62534f99eee
VolumeSize : 8
You can confirm the snapshot is completed by running Get-EC2Snapshot again and confirming the State is now completed.
Assigning Tags
If you have a lot of snapshots to manage, it can be hard to keep them all straight. To sort snapshots and categorize them for each discovery, you can also assign tags to them. Tags are a ubiquitous part of EC2 and can be used against your EBS snapshots.
You can assign tags at snapshot creation or after. To assign tags at snapshot creation use the TagSpecification parameter on New-EC2Snapshot.
For example, maybe I want to tag this snapshot has as one that I just took prior to making a production change. I’d like to create a tag called Stage and assigned it as Dev. To do that, I can create a TagSpecification object, assign one or more Tag objects to the Tags property and then define the resource type this tag will be attached to.
$tag = New-Object Amazon.EC2.Model.Tag
$tag.Key
= 'Stage'
$tag.Value
= 'Dev'
$tagSpec
= New-Object Amazon.EC2.Model.TagSpecification
$tagSpec.Tags
= $tag
$tagSpec.ResourceType
= 'snapshot'
Once I create the TagSpecification object, I can then pass that to New-EC2Snapshot.
PS> New-EC2Snapshot -VolumeId <VolumeId> -TagSpecification $tagSpec
Creating a New AMI from Snapshots
Another cool feature of snapshots is that you can create a brand new image (AMI) from a snapshot. If you have a snapshot of the root volume of a Linux only EC2 instance, you can easily create a brand new AMI from that volume snapshot. Let’s say I want to create a new image from the snapshot I just created. I can do that using the New-EC2Image command.
I’ll first need to capture the snapshot ID.
PS> $snapshotId = Get-EC2Snapshot -OwnerId self | where {$_.Tags.where({$_.Key -eq 'Stage' -and $_.Value -eq 'Dev'})} | Select-Object -ExpandProperty SnapshotId
Next, I can pass that ID as a hashtable to the BlockDeviceMapping parameter on New-EC2Image.
PS> $block = @{SnapshotId=$snapshotId}
PS>
Register-EC2Image -Name 'my_image' -BlockDeviceMapping
@{DeviceName="/dev/sda1";Ebs=$block;VirtualName='ephemeral0'}
-RootDeviceName '/dev/sda1'
Removing snapshots
If we’re finished with the snapshot, we can now remove them. Doing so is easy. Simply pipe the instance returned by Get-EC2Snapshot to Remove-EC2Snapshot. However, you may run into this error if an AMI is in use.
PS> Get-EC2Snapshot -OwnerId self | Remove-EC2Snapshot
Confirm
Are you
sure you want to perform this action?
Performing
the operation "Remove-EC2Snapshot (DeleteSnapshot)" on target
"snap-066ccd492f85192be".
[Y]
Yes [A] Yes to All [N] No
[L] No to All [S] Suspend [?] Help (default is "Y"): a
Remove-EC2Snapshot
: The snapshot snap-066ccd492f85192be is currently in use by
ami-0b353cbd967f6658f
At
line:1 char:33
+
Get-EC2Snapshot -OwnerId self | Remove-EC2Snapshot
+
~~~~~~~~~~~~~~~~~~
+ CategoryInfo :
InvalidOperation: (Amazon.PowerShe...2SnapshotCmdlet:RemoveEC2SnapshotCmdlet) [Remove-EC2Snapshot], InvalidOperationExce
ption
+ FullyQualifiedErrorId : Amazon.EC2.AmazonEC2Exception,Amazon.PowerShell.Cmdlets.EC2.RemoveEC2SnapshotCmdlet
In this case, the AMI is registered and you must first unregister it using the Unregister-EC2Image command.
PS> $myImage = Get-EC2Image -Owner self
PS>
Unregister-EC2Image -ImageId $myImage.ImageId
PS>
Get-EC2Snapshot -OwnerId self | Remove-EC2Snapshot
Summary
PowerShell allows you to manage every aspect of EBS snapshots. If the scenarios we covered here did not fit what you need to do, I suggest taking a look at all of the available EC2 snapshot commands by running Get-Command -Noun *ec2snapshot* -Module AWSPowerShell. That will give you a good idea on what’s possible with PowerShell and EC2 (EBS) snapshots.