Page 1 of 1

S3 File witgh special chars access problem

PostPosted: December 30th, 2011, 1:42 am
by colinp386
Hi,

Looking for some help in tracking down an issue with downloading files via s2Member from AmazonS3 when the file/object name contains a + or ++

If I make the file public I am able to access it via the Amazon file download URL
https://s3.amazonaws.com/unixpackages/sparc/10/gtk%2B-2.12.0-sol10-sparc-local.gz
without any issues.

If I try accessing the file via s2member with
http://domain.com/?s2member_file_download=/sparc/8/gtk%2B-2.12.0-sol8-sparc-local.gz
it errors with:
<Error>
<Code>NoSuchKey</Code>
<Message>The specified key does not exist.</Message>
<Key>sparc/8/gtk -2.12.0-sol8-sparc-local.gz</Key>
<RequestId>B4304A631D838725</RequestId>
<HostId>XXXXXXXXXX</HostId>
</Error>


if I use the + rather than %2B ie
http://domain.com/?s2member_file_download=/sparc/8/gtk+-2.12.0-sol8-sparc-local.gz
I get:
<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
<StringToSignBytes>47 45 54 0a
<snip --
||
snip >
6e 73 65 2d 65 78 70 69 72 65 73 3d 46 72 69 2c 20 32 33 20 44 65 63 20 32 30 31 31 20 30 32 3a 34 31 3a 30 35 20 47 4d 54</StringToSignBytes>
<RequestId>8A90E89632DA6ADB</RequestId>
<HostId>XXXXXXXXX</HostId>
<SignatureProvided>XXXXXXXXXSignatureProvided>
<StringToSign>GET 1325212895
/unixpackages/sparc/8/gtk-2.12.0-sol8-sparc-local.gz?response-cache-control=no-cache, must-revalidate, max-age=0, post-check=0, pre-check=0&response-content-disposition=attachment; filename="gtk -2.12.0-sol8-sparc-local.gz"&response-content-type=application/x-gzip&response-expires=Fri, 23 Dec 2011 02:41:05 GMT</StringToSign>
<AWSAccessKeyId>XXXXXXXXX</AWSAccessKeyId>
</Error>


I am also using . Donovan Schonknecht's S3php class to bring back the file attributes and that also fails on these files, I'm trying to figure out if its a generic AmazonS3 issue of it it's a problem with a lack of URL encoding by the plugin/class. I am on a version of PHP that supports encoding.

Any help much appreciated, as its not easy for me to have to remove all the +'s from the file names.

Colin

Re: S3 File witgh special chars access problem

PostPosted: December 30th, 2011, 2:07 am
by colinp386
Ok after digging around a bit more I think I see what's happening

The string is being un-encoded and its changing the plus to a space which is what it's supposed to do.
As when you encode a string a space is converted to a plus
I see that now in the report of the Key <Key>sparc/8/gtk -2.12.0-sol8-sparc-local.gz</Key>
as it was gtk+-2.12

Any ideas?

Re: S3 File witgh special chars access problem

PostPosted: December 30th, 2011, 4:40 am
by Cristián Lávaque
I'll email Jason about this.

Maybe you should remove the plus sign from the file name to avoid this problem in the meantime. :)

Re: S3 File witgh special chars access problem

PostPosted: December 30th, 2011, 1:01 pm
by colinp386
Cristián Lávaque wrote:I'll email Jason about this.

Maybe you should remove the plus sign from the file name to avoid this problem in the meantime. :)


Tell me about it! LoL I think that's what I may have to do, its a royal PITA to find and replace 80 files out of 14,000 on S3 :cry:

Thanks for the escalation. I blame those FOSS communities for creating packages with friggin + signs in the package name! Argh!!

Re: S3 File witgh special chars access problem

PostPosted: December 30th, 2011, 5:51 pm
by colinp386
Incorrect information deleted by poster.

Likely if there is an issue with any file access issues we will find them LoL

Best

Colin

Re: S3 File witgh special chars access problem

PostPosted: December 30th, 2011, 9:41 pm
by Jason Caldwell
Thank you Colin. I'm taking a look now to see if we can patch this in for the next update.

Re: S3 File witgh special chars access problem

PostPosted: December 30th, 2011, 10:06 pm
by colinp386
No problem. Though I have some concerns about the code I provided (written by a colleague) as while they appear to work form him, I'm not seeing that it's fixed the issue.

Re: S3 File witgh special chars access problem

PostPosted: December 30th, 2011, 10:25 pm
by Jason Caldwell
Not to worry, we carefully review all changes such as this.
~ I appreciate your efforts though, it really helps!

Re: S3 File witgh special chars access problem

PostPosted: December 31st, 2011, 11:40 am
by colinp386
Ok we worked up a solution to the problem, in addition to adding the following to
Class: c_ws_plugin__s2member_files_in
Method: amazon_s3_url
Code: Select all
$file = rawurlencode($file);


To make downloads work we made a simple user function:
Code: Select all
add_shortcode('url-encode', 'wp_url_encode_wrapper');
   function wp_url_encode_wrapper($atts, $value){
     return urlencode($value);
}



And added it to the get file url ie.

Code: Select all
<a href="/?s2member_file_download=/sparc/10/[url-encode]gtk+-2.12.0-sol10-sparc-local.gz[/url-encode]">gtk+-2.12.0-sol10-sparc-local.gz</a>


So while it does not fix what looks to be an issue with the plugin (I reserve the right to be wrong ;) ) it does give us a work around

This is really a cludge and a better fix is needed.
Note we make a mistake originally and encoded the basename as well, not realizing it was the return filename, oops :-)

Re: S3 File witgh special chars access problem

PostPosted: January 2nd, 2012, 9:45 pm
by Raam Dev
Colin,

Thank you very much for posting this workaround! :) I can see that shortcode coming in handy.