Error with Amazon S3 storage function in Newest Version

[cclambie]

Hi S2 Member Users,

I was excited to try out the new Amazon S3 storage tool, and have done, but unfortunately I have ran into an error.
You can try clicking on the "play" button on this test page
http://tinyurl.com/3ogvsy7#truesecrets.com.au which will take you to http://truesecrets.com.au/test-page/ to see the error

<Error>
<Code>PermanentRedirect</Code>
−
<Message>
The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
</Message>
<RequestId>B90F3F336BAD5CC1</RequestId>
<Bucket>truesecretsaudio</Bucket>
−
<HostId>
09LyXaVimQ4QmcrUYmxmmrKmkIwdTHLED9FPLyviYYOiS4QbBODBWEjOZ9cU+yvR
</HostId>
<Endpoint>truesecretsaudio.s3.amazonaws.com</Endpoint>
</Error>


Any thoughts?

[drbyte]

Hi

Edit ACL

Your bucket should have read only (owner) and your files should have read Everyone

I use CloudBerry S3 explorer pro...there is a free version though or you can use S3 Firefox organizer

Sam

[cclambie]

Hi Dr Byte,

If I set the Files to "everyone" I may as well not use S2 member to protect them? If you know the name of my bucket and the file, you can just simply download the file.

I thought this script would create a temporary link to the file, that expires.
I downloaded and used the Cloudberry app, it has an option to create a temporary link, obviously using Amazon S3 API, I thought this is what s2Member plugin was going to be doing?

This is it - lasts 24 hours (till June 7 local Australian EST)
https://truesecrets1.s3.amazonaws.com/A ... Q7FsozY%3d

I am still getting the error when I use the S2Member app and the file is set to public...

Thoughts?

C

[drbyte]

Well, no body can browse your files if the folder is set to read only (owner)...to access the files, they need to be set to Everybody otherwise you will get an error.

Having said that, S2m will deny access to files if it's set under some level (1,2,3...)...and on top of that you can use CNAME so no body can tell what is the name of your Bucket. For Example..your bucket name is xyz.s3.amazonaws.com you can use CNAME to have it set to http://www.xyz.yoursitename.com

For Example:
First create a bucket in Amazon S3 with exactly the same domain name / sub domain name. I wanted the URL to be imgs.site.com and so my bucket name in Amazon is also imgs.site.com.

Now go to your domain DNS. In my case site.com is registered at GoDaddy and so I go to DNS control for the Domain and setup a new CName for my domain i.e. imgs and target that to the bucket of Amazon s3 i.e. imgs.site.com.s3.amazonaws.com. Setting the target to s3.amazonaws.com also works but results in extra HTTP redirects.

Wait for the DNS to propagate but for new CName the change is instant. Now upload everything under the bucket and if you allow public read for the uploaded content you will be able to view your uploaded files in the browser.

You will be fine as long you keep you bucket name hidden...use the CNAME. Or you can use get pre-signed urls.. and have it expired within 24 or more...

s2M S3 works exactly as you'd have the files inside your plugin folder...

You need to secure your bucket first...it's set to read to everybody

Sam

[cclambie]

Hi Sam,

Ok, before I try the Cname thing, which I will eventually sure, but lets just get a link to work first.

I fixed the bucket so it should be owner only now.
ACL settings on the two files are different, one with read everyone, one with read owner only.

Still no joy through S2?
I am hosting on IIS, is this an issue?

Thanks
C

[drbyte]

Aha, well if you are on a windows box then you need to take ownership of the root directory...IIS is not that great with WP..they have issues especially with pretty links

Do you have a live site or local?

Is all other S2M download fields correct?

[cclambie]

I have both a local and live site.
The links above are live sites, as you obviously can't see my local site
Doesn't work on either...

I have had issues with s3flowshield for a similar reason, .htaccess redirect rule issues, does S2 use these too?
I really want to resolve this problem.

[drbyte]

are you using US Standards? if you create a bucket that is not "US Standard", you cannot use path-style syntax in the bucket address


Have you tried using http://www.site.com/?s2member_file_down ... e-file.zip

Sam

[Jason Caldwell]

Investigating now.

Sam, I think you're correct on the "US Standard", I'm taking a look now to see if there is something we can do to assist in this regard. As for ACLs, here is how your Bucket should be protected on the Amazon S3 side. ( this example shows permissions configured with S3 Fox, a plugin for Firefox )

And here's what it looks like in the AWS management console.

[cclambie]

That was it!
I created a bucket using US-Standard and that fixed it right up

I am in Australia, so I was using Singapore.
No need to open the bucket up for read, it all works as planned now.
Excellent work! Thanks

Might be an idea to add these two points into the help
- Bucket must be US-Standard
- You must install a Plugin like PHP Exec or similar to enable the download_key function to work

[Jason Caldwell]

Thanks for reporting back.

Yea, I'm having s2Member updated to support all regions.
s2Member v110606 has just been released ( available now ).
http://wordpress.org/extend/plugins/s2member/

Here is a Changelog excerpt.

* **(s2Member). Amazon® S3 bug fix.** s2Member has been updated to support all regions, and not just the `US Standard`. If you were getting errors in the previous release, related to Amazon® S3 storage locations being accessed improperly, this release of s2Member v110606 should resolve the issue for you. To put it simply, s2Member now connects to your S3 Bucket using `[bucket].s3.amazonaws.com`, instead of `s3.amazonaws.com/[bucket]`, making s2Member more compatible in this regard; and as suggested by this documentation. This also improves the User experience, since your Bucket name will appear in the host name of the download, instead of just `s3.amazonaws.com`. For further discussion, please check [this thread](http://www.primothemes.com/forums/viewtopic.php?f=4&t=10054).

[cclambie]

Excellent!

I have upgraded and tested the new version with same successfully result using bucket in Singapore

[cclambie]

Not sure what has happened?
I have tried every variation, and I have used and I simply can't get it to work.
I am getting the following error
URL: http://truesecretsus.s3.amazonaws.com/A ... 9VWYRvY%3D

Code: Select all

<Error>
<Code>AccessDenied</Code>
<Message>Request has expired</Message>
<RequestId>9C88278B3B32DE86</RequestId>
<Expires>2011-06-10T05:03:40Z</Expires>
<HostId>
xZmgzlDVd7GDoHU+q7bt9Q/c9Z5B/zQ9CzyelSnCUfLMauzcPHMSXn9l0xl9hdTV
</HostId>
<ServerTime>2011-06-10T05:04:35Z</ServerTime>
</Error>

[drbyte]

It's simply telling you that the link to that item has expired. (2011-06-10T05:03:40Z)
You need to generate a new link. Do remember that once expired it will not auto regenerate a new one unless you are using some other plugin

Now http://truesecretsus.s3.amazonaws.com/A ... Master.mp3 is still access denied. if you don't mind me asking, why using the S3 bucket name instead of your site?

Sam

[cclambie]

ok. I have set up CNAME now

I think I have resolved all the access denied errors, so now I just need to resolve why I can't get an external program to stream this file.
See other Forum

[kangenwater]

Hi Jason,

Is it possible to have a video explaining Amazon S3 storage functionality from beginning to end?

I have spent so much time configuring S2Member Amazon S3 and still getting errors - with S2Member V110710 (latest as of date of post). I counterchecked my access key and secret key to make sure it is not in error.

Temporarily I have put the ACL to Everyone in Read, Write, FullControl (which defeats the purpose).

On another note, are we able to create subdirectories in the Amazon S3 bucket that corresponds to the Level 0, Level 1, Level 2, Level 3, and Level 4? like:

Sub-Directory: /s2member-files/access-s2member-level0
Sub-Directory: /s2member-files/access-s2member-level1
Sub-Directory: /s2member-files/access-s2member-level2
Sub-Directory: /s2member-files/access-s2member-level3
Sub-Directory: /s2member-files/access-s2member-level4

[Cristián Lávaque]

s2Member 110731 had an update to the S3 integration, not sure if it'll help with the trouble you're having:

https://wordpress.org/extend/plugins/s2 ... changelog/

(s2Member) Compatibility. In cases where a site owner integrates Amazon® S3 file storage for protected File Downloads, with an invalid Bucket name which contains mixed caSe ( that's a no-no anyway ), s2Member will now convert the Amazon® S3 connection URL to s3.amazonaws.com/[BUCKET], instead of the default [bucket].s3.amazonaws.com. This is required for invalid Bucket names which use mixed caSe. The only way for Amazon® S3 communication to work with these invalid Bucket names, is with a sub-folder URL. See docs on Bucket names.

[kangenwater]

I am using the latest version of s2member and my bucket name is valid - no mixed caSe.

I am still getting this error:

Code: Select all

<Error>
   <Code>AccessDenied</Code>
   <Message>Access Denied</Message>
   <RequestId>C02D810644E351C4</RequestId>   
   <HostId>
      7ErUJHPHu5lee8BrX9FCbEAUMsFcmMBkbYb6e2hB6rSbJSYnNZNdPviSTvoia1t
   </HostId>
</Error>


I put the directory ACL based on owner Read, Write, and Full Control.
Everyone and Authenticated Users have no access based on the description above.
I also tried giving Authenticated Users - Read, Write, and Full Control - but that does not work either?
Please advise...

[Jason Caldwell]

Thanks for reporting this important issue.

Although we have MANY site owners working with s2Member/Amazon S3 without any problems, I recently had another report regarding this same exact error. We never did figure out the problem, but the site owner told me that he changed the name of the file in his S3 Bucket to something different and it seemed to make the problem go away. So it seems that it could have something to do with the Amazon S3 side of things.

If all else fails, please send me a Dashboard login through this private contact form and I'll try to diagnose the problem again on your installation. Hopefully we can get to the bottom of this. See: http://www.s2member.com/contact/

[keywordcentrics]

Hi Jason,

I have had the same issue as far as the Amazon S3 'Access Denied'.
I have followed your instructions I believe to a "T."

Inside of the s2Member plugin in WordPress, I put my bucket name
as "keywordcentrics" and that alone. I also put the Access Key and Security Key.

I changed the name of the files because I thought that might have been
the issue hearing about the other user. I changed it to a lowercase name
"kcengineeringprt1.mp4", and the problem has remained the same.

I do know I can change the "ACL", but that would defeat the purpose
I think of securing the content as much as possible unless the person buys.

I don't know what to do. I am almost done with everything and just need to get
this figured out.

~Gary

[Jason Caldwell]

Thanks for reporting your experience. If one of you can send a Dashboard login to help us reproduce this, that may help, as we've been unable to reproduce the issue on our end so far. This seems to be server-specific, or possibly Amazon S3 account-specific. We're just not sure yet. While the majority of s2Member installations are unaffected by this, I have seen reports like these come in over the last two weeks and we'd like to get to the bottom of it. If you can supply a Dashboard login where this behavior can be reproduced, please send me the information privately through this contact form: http://www.s2member.com/contact/

Also, please be sure to test against s2Member v110731+, as we did release a tweak that accommodates Amazon S3 Buckets with mixed caSe, as seen in the changelog excerpt below:

* **(s2Member) Compatibility.** In cases where a site owner integrates Amazon® S3 file storage for protected File Downloads, with an invalid Bucket name which contains mixed caSe ( that's a no-no anyway ), s2Member will now convert the Amazon® S3 connection URL to `s3.amazonaws.com/[BUCKET]`, instead of the default `[bucket].s3.amazonaws.com`. This is required for invalid Bucket names which use mixed caSe. The only way for Amazon® S3 communication to work with these invalid Bucket names, is with a sub-folder URL. See docs on [Bucket names](http://docs.amazonwebservices.com/AmazonS3/latest/dev/index.html?BucketRestrictions.html).

[keywordcentrics]

Thanks for getting back with me so soon. I know it's a Saturday. I emailed the information you need.

Appreciated!
Gary

[Jason Caldwell]

Thank you for the follow-up.

Thanks for getting back with me so soon. I know it's a Saturday. I emailed the information you need.

Appreciated!
Gary

OK. In your case, you simply have a configuration problem.

First, File Downloads must be enabled on your site, by going to s2Member -> Download Options -> Basic Download Restrictions, and configuring at least one of the Membership Levels there.

Secondly, you were attempting to use a direct link to the protected file, via CloudFront:

Code: Select all

http://xxxxxx... .cloudfront.net/kcengineeringprt1.mp4

* this is BAD

Instead, please create your links as instructed by s2Member:

Code: Select all

http://xxxxx... /?s2member_file_download=kcengineeringprt1.mp4

* I ran a test on this file from your site, and everything appears to be working properly when the link is formulated with s2Member's instruction. Please let us know if you continue to have trouble.

[keywordcentrics]

Hi Jason,

I didn't know that 'Cloudfront' was bad to use.
I guess I was just looking for a way to distribute it,
as well as stream it.

What I mean is: I am trying to make it where my members can download it,
but I have seen on other sites where they have also embeded somehow the
s3 amazonaws file on their site to stream at the same time.

Not sure if that makes sense. I am trying to find how you actually made the link
for me. You all did a very good job in the the support videos. I will be getting s2MemberPro
soon, and I wanted to know if I would lose my settings and everything from going to the free version?

Gary

[Jason Caldwell]

I didn't know that 'Cloudfront' was bad to use.
I guess I was just looking for a way to distribute it,
as well as stream it.

Well, CloudFront is not bad to use. I didn't mean to imply that. In fact, CloudFront is a CDN ( Content Delivery Network ) in the truest sense. However, Amazon S3 by itself is also a CDN, just not quite as widely supported at CloudFront in terms of "reach" across the world with better speeds. This is why CloudFront exists.

That being said, the Amazon S3 API does not allow for protected files to be served through CloudFront anyway, only through the Amazon S3 authenticated link API does this function with s2Member. So using CloudFront is recommended for unprotected files, but NOT for protected files served by s2Member.

You will find details on how to formulate your links, inside your Dashboard, under:
s2Member -> Download Options -> Basic Download Restrictions.