Category Archives: SDK

My experience using Amazon’s CloudFront as CDN – Part II

IMPORTANT: There is a previous post on this topic where I’ve introduced some interesting concepts about using CloudFront as CDN. I recommend you to read it before reading this one.

Distributing private content using CloudFront

One of the most important differences between CDNs are the way that they allow you to distribute private content. There are several options to distribute private content from a CDN, but depending on the number of private files to distribute is important the way that you can do it.
CloudFront lets you generate temporal signed URLs to allow third parties access private content through these links. CloudFront signatures can be generated using the PHP SDK and the Amazon account credentials without any need of external API calls. If your webpage contains more than one private file to load, usually avoiding external calls can improve your response time. It always depend on the application behavior but adding a latency time more than twice to your response time does not seems a good idea at all.

cloudfrontlogo

OAI’s concept

The easier option when defining the data origin for a CloudFront distribution is to chose an S3 bucket. It lets you to manage your files with your favorite S3 files manager (I have just discovered Dragondisk, and it is extremely useful for managing your Amazon buckets) without further configurations.
Applying this setup will distribute your static files through the CloudFront locations as long as you defined these files as public. If you define some S3 objects as private, CloudFront daemons won’t be able to read the object content and they won’t be able to replicate it across their distributed sites. At this point you will need to start thinking about OAIs.
OAIs are virtual users that can be allowed to access S3 objects without defining these objects as public (they will be kept as private objects but defined OAIs will be able to access them). Creating OAIs, and granting them read access to S3 files are simple operations but they are (IMHO) not clearly explained in the AWS official documentation.Using the  official AWS SDK for PHP, the OAI set up is as easy as:

$cf = new AmazonCloudFront();
$cf->set_keypair_id(‘aws_keypair_id’);
$cf->set_private_key(‘aws_private_key’);

$res = $cf->create_oai(‘referer’);

You will need to remember the “referer” value for future operations. It isn’t possible to create two different OAIs with the same referer so, you will need to use different values for different OAIs.

Setting S3 object permissions

Grating OAIs access to S3 objects is another easy task as long as yo know what to do. Although it is a mandatory step to have a private distribution, in my humble opinion, it isn’t enough highlighted in the official documentation (anyway you will find it if you know what you are looking for).Once again, using the official SDK for PHP:

$s3Instance = new AmazonS3();

$custom_acl = array(
array( ‘id’ => ‘oai_canonical_id’, ‘permission’ => AmazonS3::GRANT_READ ),
array( ‘id’ => ‘aws_user_canonical_id’, ‘permission’ => AmazonS3::GRANT_FULL_CONTROL));

$res = $s3Instance->set_object_acl(‘bucket_name’, ‘s3_object_name’, $custom_acl);

 

Note that you will need to find the OAI and the AWS user canonical IDs, I found them listing OAIs and S3 objects properties using the SDK. Canonical IDs are hashes that seem to be unique all across the AWS services (it is just a personal evaluation, I’ve not confirmed this with anyone from Amazon) and let you grant OAIs access to S3 objects.

Creating a private content distribution

This is the final step to have a private (signed) content CloudFront distribution. When I was working with CloudFront, it wasn’t possible to set up a distribution like this using the AWS web console so, I will show how to do it using the SDK:$cf = new AmazonCloudFront();
$cf->set_keypair_id($yourAWSKeypairID);
$cf->set_private_key($yourAWSPrivateKey);

$opts = array(
‘Enabled’ => true,
‘OriginAccessIdentity’ => ‘oai_id’,
‘TrustedSigners’ => array(‘Self’));
$res = $cfInstance->create_distribution(‘bucket-name’, ‘dist_name’, $opts);

Note that you need to configure the AmazonCloudFront object with additional security credentials and that you will need to retrieve the OAI id to create the distribution.

Once the distribution is created, all objects within will need to be retrieved using a signature from the AWS user (that’s the reason why the TrustedSigners property have ‘Self’ as value). But this is something that I will explain in the next CloudFront-related post.

My experience using Amazon’s CloudFront as CDN – Part I

Preparing a website for more than 4 million users per month can be a hard task to achieve. Obviously, you cannot start a project from scratch pretending to support this amount of users the first week (if you can support it economically… do you want to hire me? :) ) but maybe you can predict that you will reach a huge amount of visits before the end of the current year.

I’m not going to describe step by step how I’ve tried to support a large amount of users hitting a web application -it’s something that is not written on stone because all applications have different behavior and processes and maybe solutions that worked on my project can ruin everyone else applications- but I want share my experience while dealing with Amazon’s CloudFront.
This deep desire of sharing my thoughts, troubles and successes with CloudFront is due to the underneath complexity of this CDN system. Yes, it seems to be a very basic and simple distribution service and, sometimes, it may lead to think that you cannot go further with the Amazon’s CDN solution but… give it a try. As always, there is room to grow for CloudFront but, let’s see…
NOTE: The project that I’m using while writing this post (or set of posts) is developed in PHP, so I tried to keep my tools as close as possible to this programming language.
cloudfrontlogo

The Beginning

The first approach to CloudFront is always using the console provided directly by Amazon. This console allows us to see all our distributions and perform configuration updates on them. It’s easy to use, and easy to get on with but, believe me, you will need more tools than this to have your distributions under real control.
The easiest way to have a distribution up and running is linking it to an existing S3 bucket using only the download configuration (not the streaming one). It costs less than a minute to set up and its results are very competitive CDN resources.
Although the web console has useful features like creating, updating basic parameters, enabling/disabling and deleting distributions, it is only the spark that can light up a candle to guide us to more complex distribution deployments.

Getting into the dark

If your aim is just to have a basic CDN to distribute public content such as design images, videos and stylesheets stop reading now. Otherwise, you can keep on reading and discover what I was able to find behind the scenes.
A usual problem using CDNs for static content distribution is that hotfixes can be allowed and deployed as long as they do not impact on the static resources. Due to the distributed architecture of the CDNs infrastructures it’s really difficult to update static resources when you are short in time (but there are no IT projects with scheduling issues, are there?).
And guess what… Amazon provides a process to speed up this kind of operation. Using the official Amazon WebServices PHP SDK there is a way to delete all the copies of static resources on all geographic locations. This allows Amazon distributed datacenters to download the new version of modified resources. This process is called invalidation and must be used carefully.
A short sample of an invalidation code:

$cloudfront = new AmazonCloudFront();
$cloudfront->set_keypair_id(‘your_keypair_id’);
$cloudfront->set_private_key(‘your_private_key_content’):

$unwanted_files = array(‘file_to_invalidate1′, ‘file_to_invalidate2′, ‘…’);
$cloudfront->create_invalidation(CLOUDFRONT_DISTRIBUTION_ID, ‘Invaliation Name’, $unwanted_files);

NOTE: This code snippet is only valid if you have previously installed the PHP AWS SDK.

And that’s all for today, in the coming weeks I will try to write a post describing the mandatory steps to set up a CloudFront distribution for private content (or not strictly private but signed).

Hope it helps! :)