Solving AWS::S3::NoSuchBucket Error When Using Paperclip

September 9, 2011   ·   By Sheldon Conaty

Very often when working on server side Ruby on Rails projects we need to support file uploads, such as pictures. To provide this support our gem of choice is Paperclip. We've used Paperclip for years and found it a huge improvement over attachment_fu which we used prior to it. However, in our last project it threw us a curve ball.

As we've done in dozens of previous projects we created an S3 bucket, configured Paperclip to use it and then started uploading files. However, uploads to the bucket failed with the following error appearing in the logs…

NameError (uninitialized constant AWS::S3::NoSuchBucket)

Petty strange since the bucket did exist.

So I used my trusty CyberDuck S3 client to connect to the bucket and upload some test images. I then tried to access these with the URL Paperclip was generating for them. For example http://s3.amazonaws.com/topsecretbucket/pictures/entity/3/pretty+picture+1.JPG?1311614390

The Amazon server returned an illuminating response…

<Error>
  <Code>PermanentRedirect</Code>
  <Message>
    The bucket you are attempting to access must be addressed using
    the specified endpoint. Please send all future requests to this endpoint.
  </Message>
  <RequestId>2EACG096167A1212</RequestId>
  <Bucket>topsecretbucket</Bucket>
  <HostId>
    dV/835QeLLt/g/t/Lhl+SCTVTGV3uo801DBUzw+tq85Nc+gTPDTDEiq3+oTwmKcM
  </HostId>
  <Endpoint>topsecretbucket.s3.amazonaws.com</Endpoint>
</Error>

Notice the endpoint Amazon recommend has the bucket name as a subdomain and not as part of the path.

Eureka!

The root of the problem was my bucket, which was configured to be located in the EU and not the US, as we've used in all our previous projects. Once we recreated a "US Standard" bucket the problem was solved. An examination of the Paperclip source revealed it only supports US based buckets.

As stated in the Amazon Simple Storage Service Developer Documentation...

Important

Amazon S3 supports virtual-hosted-style and path-style access in all Regions. The path-style syntax, however, requires that you use the region-specific endpoint when attempting to access a bucket. For example, if you have a bucket called mybucket that resides in the EU, you want to use path-style syntax, and the object is named puppy.jpg, the correct URI is http://s3-eu-west-1.amazonaws.com/mybucket/puppy.jpg. You will receive a “PermanentRedirect” error, an HTTP response code 301, and a message indicating what the correct URI is for your resource if you try to access a non US Standard bucket with path-style syntax using:

• http://s3.amazonaws.com
• A different Region endpoint than where the bucket resides, for example, http://s3-euwest-1.amazonaws.com and the bucket was created with the location constraint of Northern-California

The S3 module in Paperclip is hardcoded to use http://s3.amazonaws.com and so can only reference US based buckets.