Sunday, October 20, 2013

Creating Backups in AWS Glacier with CrossFTP

Amazon Glacier is a storage service that is priced/optimized for long-term data archival and backup.  At the current time, storage charges are $0.01 per gigabyte per month (eastern region) with a number of minor ancillary charges.  Some basic operations can be performed through the AWS Console, but most require the use of either a specialized FTP client or custom code.  I opted for the quick and easy route, and found CrossFTP after researching a few available options.  They already had a tutorial posted on their blog, and the snapshots below are just my own notes from following that process.

After installing the CrossFTP Pro software, the following steps were used to setup a new site, create a Amazon Vault, and upload a sample file (note that as soon as the file upload completes to Amazon's servers, your charge meter begins to tick).

First, a little house-keeping.  In the CrossFTP Site Manager (menu Sites | Site Manager), create an organizational folder to hold any AWS sites (not required, I just can't stand an unstructured list):


To create the site listing, you will need a two pieces of information in advance:


Press 'Close' to save the site.  It will appear in the CrossFTP site list, and connecting is just a double-click away



Amazon uses the term "Vaults" to refer to the top level directories/folders within your Glacier account, and the only apparent reason for that resides in the brains of marketing personnel; as far as I can tell, they are just directories.  I created a directory called "Backups" by using the "plus folder" icon in CrossFTP
  


To verify the operation worked, I logged into the Amazon console and displayed my list of Glacier vaults in the eastern region.  The new backups folder/Vault was present and accounted for



The last step involved actually putting a file online.  Note that since this is where the primary charging from Glacier originates, you'll want to ensure you upload things you actually care enough to pay for on a monthly basis.  In this case, I selected an old photo, renamed it for illustrative purposes, and uploaded it.




To disconnect from the site in CrossFTP, press the "planetary X" button above.

Next on my radar is to spend some time determining the best file format to use for storage on Glacier.  Since charging occurs by the byte, clearly a compressed format would be desirable, and for the sake of security, some kind of encryption would also be in order.  However, introducing compression and encryption means that the loss of even 1 bit of data could be catastrophic.  Per documentation, Amazon claims an average annual durability of 99.999999999% and performs at-rest encryption using AES-256.  However, for those of us who are truly paranoid, a few more steps are probably in order.  More investigation is needed, but parity archive files via Parchive may be part of the answer.

3 comments:

  1. Finding the time and actual effort to create a superb article like this is great thing. I’ll learn many new stuff right here! Good luck for the next post buddy..
    AWS Training in Chennai

    ReplyDelete
  2. Nice information thanks for providing keep update AWS Online Training

    ReplyDelete
  3. Great post I must say and thanks for the information. Education is definitely a sticky subject. However, is still among the leading topics of our time. I appreciate your post and look forward to more. 4 ps of marketing

    ReplyDelete