Validated bucket name in S3 form uploader depends on S3 URL

[NamespaceValentine]

I am attempting to correct a bug when uploading to s3 through Cloudfront using ie9. I have set up the blank page required, tied it in, allowed cors on both the client and s3. My error is happening during the GET that occurs after the POST to cloudfront. The GET shows it is correctly rerouting to my blank page, the query string has the following arguments:
bucket - correct bucket
key - correct key
etag

The GET responds with a 200, no response body, and no request body that I can see. I get the following two errors in my console:
Response from AWS included an unexpected bucket or key name.
A success response was received by Amazon, but it was invalid in some way.

I am unsure how to troubleshoot from here, there is no documentation I can find on either error, and I have tried very hard to ensure I followed all instructions on ie9 compatibility (http://blog.fineuploader.com/2013/08/16/fine-uploader-s3-upload-directly-to-amazon-s3-from-your-browser/#success-redirect). The uploads work on ie10+ and all other browsers.

[NamespaceValentine]

My options

{ element: document.getElementById('sr-fine-uploader'), template: 'qq-template-s3', request: { endpoint: CLOUNDFRONT DISTRO HERE, accessKey: "REDACTED" }, cors: { expected:true, allowXdr:true, allowCors: true }, chunking: { enabled:true, partSize: 31457280, concurrent: { enabled:true } }, objectProperties: { acl: 'bucket-owner-full-control', bucket: BUCKET NAME HERE, host: BUCKET NAME FORMATTED AS HOST HERE, region: "us-west-2", serverSideEncryption: true, key: function(id){ return keyPrefix() + s3Uploader.getName(id); } }, signature: { endpoint: '/signature', //Correctly signs in all cases version: 4 }, retry: { enableAuto: true, autoRetryNote: document.getElementById('retry-text').name+" {retryNum}/{maxAuto} ..." }, text: { waitingForResponse: document.getElementById('waiting-for-response').name }, validation: { sizeLimit: 1073741824 }, messages: { sizeError: "You can upload a file or multiple files. Individual file size cannot exceed 1GB." }, iframeSupport: { localBlankPagePath: "/fineUploaderBlank.html" } }

[NamespaceValentine]

I think I have traced the issue. The file is getting correctly uploaded, but we disallow people to get at the file without a signed url. Fineuploader seems to be checking to see if the file was uploaded, but it cannot see the file, therefore it considers it a failure. Is there a way to turn off the test, or to add some business tier logic that tests the upload by requesting the signed url?

[NamespaceValentine]

This may be an inaccurate understanding of what the tests are that you are running, but that is my best guess based on this documentation.
"To be absolutely sure, Fine Uploader S3 examines some parameters in the iframe’s URL (such as the bucket and key) to ensure that the response refers to the correct file."

[rnicholus]

CORS/XDR is completely irrelevant here, unless your signature server is on a different domain. If you are seeing an issue where Fine Uploader reports an unexpected response to an upload request, you'll need to provide the complete headers for the request and response payload/headers as well.

[rnicholus]

S3 will redirect to the localBlankPath endpoint you specified of the upload succeeds. If this is not happening, then the upload has not succeeded or this endpoint is not valid. If it is happening, then there is some issue with with specified bucket in the redirect URL. In order to troubleshoot I'll need the relevant request/response info.

[NamespaceValentine]

The redirect is happening, that is the GET that is failing. The form submits correctly because the file makes it to S3, but the UI claims that the file upload fails, and so the user sees an error. Here are the headers for the blank page GET:
Request headers:
Request: GET /fineUploaderBlank.html?bucket=[correct bucket here]&key=[expected key here, url encoded]&etag=[urlencoded etag]
Accept: text/html, application/xhtml+xml, /
Referer: [referring uri here]
Accept-Language: en-us
User-Agent: Mozilla/5.0 (compatible; MSIE 9/0; Windows NT 6.0; Trident/5.0)
Accept-Encoding: gzip, deflate
Host: [our url, same dns as the blank html page]
Connection: Keep-Alive
Cache-Control: no-cache
Cookie: [several cookies that we normally pass around our site on requests here]

Response headers:
Response: HTTP/1.1 200 OK
Cache-Control: max-age=0, no-cache
Content-Type: text/html;charset=UTF-8
Date: Wed, 24 Feb 2016 18:05:42 GMT
Strict-Transport-Security: max-age=15552000
Connection: keep-alive

I see no request body in this request. In case you need it, here are the headers for the request that causes the redirect (the POST to cloudfront):
Request headers:
Request: POST /HTTP/1.1
Accept: text/html, application/xhtml+xml, **
Referer: [referring uri here]
Accept-Language: en-us
User-Agent: Mozilla/5.0 (compatible; MSIE 9/0; Windows NT 6.0; Trident/5.0)
Content-Type: multipart/form-data; boundary=-----------------------------7e02bb1e10172
Accept-Encoding: gzip, deflate
Host: [cloudfront dns]
Content-Length: 1348053
Connection: Keep-Alive
Cache-Control: no-cache

Request Body:
-----------------------------7e02bb1e10172

Content-Disposition: form-data; name="key"

id-2775747901/1MBzipFile.zip

-----------------------------7e02bb1e10172

Content-Disposition: form-data; name="success_action_redirect"

[URI for blank page here]

-----------------------------7e02bb1e10172

Content-Disposition: form-data; name="x-amz-server-side-encryption"

AES256

-----------------------------7e02bb1e10172

Content-Disposition: form-data; name="acl"

bucket-owner-full-control

-----------------------------7e02bb1e10172

Content-Disposition: form-data; name="x-amz-meta-qqfilename"

1MBzipFile.zip

-----------------------------7e02bb1e10172

Content-Disposition: form-data; name="x-amz-algorithm"

AWS4-HMAC-SHA256

-----------------------------7e02bb1e10172

Content-Disposition: form-data; name="x-amz-credential"

[AWS Access Key Id here]/20160224/us-west-2/s3/aws4_request

-----------------------------7e02bb1e10172

Content-Disposition: form-data; name="x-amz-date"

20160224T181030Z

-----------------------------7e02bb1e10172

Content-Disposition: form-data; name="policy"

[Policy here]

-----------------------------7e02bb1e10172

Content-Disposition: form-data; name="x-amz-signature"

[Signature here]

-----------------------------7e02bb1e10172

Content-Disposition: form-data; name="file"; filename="1MBzipFile.zip"

Content-Type: application/x-zip-compressed

[Binary File Data Not Shown]

---------------------------7e02bb1e10172--
Response headers:
Response: HTTP/1.1 303 See Other
Content-Length 0
Connection keep-alive
Date: Wed, 24 Feb 2016 18:05:43 GMT
x-amz-server-side-encryption: AES256
ETag: "[etag here]"
Location: [the https endpoint for the empty html page along with the same query params as the GET request]
Server: AmazonS3
X-Cache: Miss from cloudfront
Via: 1.1 ea961b1c94972f8feffbb14a32ffee07.cloudfront.net (CloudFront)
X-Amz-Cf-Id: L37o25JSoD1PO_C-OwH5daTaCw8EcUODgJl7yPCiyFIOuPwTxQ==

[NamespaceValentine]

I have found the issue, but do not know how to resolve it. The bucket recieved back from AWS is the correct bucket, but the bucket it is expecting is the cloudfront endpoint. Meaning in the function isValidResponse, qq.s3.util.getBucket(endpoint), which is what bucket gets set to, is grabbing the request endpoint, but is trying to compare it to the bucket that the endpoint is referencing. Should this not be being set by the objectProperties.bucket? If it was, these two would match.

[rnicholus]

Yes, this does appear to be a bug, caused by some rigid logic in the S3 form upload handler. Like other S3 code in Fine Uploader, the form upload handler needs to use a shared function that determines the proper bucket given a specific file ID. I'll look into fixing this in 5.5.2. If you could make yourself available to test the fix, that will make this a bit smoother.

[NamespaceValentine]

I am more than willing to make myself available. Just let me know what you need.

[rnicholus]

Great! Today is my last day in the office until March 2nd, so I'll start working on a fix shortly after I return.

[NamespaceValentine]

Thank you, in the meantime I'll just fork it locally and ignore the check, when the fix is available I'll swap back.

[rnicholus]

s3.fine-uploader-5.5.2-1.zip

Please see the attached build of Fine Uploader S3 5.5.2-1 with the proposed fix for this issue. Let me know if this works for you (and does not cause any further issues).

[NamespaceValentine]

The fix worked for our use case. Thank you very much.

[rnicholus]

Thanks for the confirmation. I'll release this fix as part of 5.5.2 on Monday or Tuesday.

[rnicholus]

Fixed in 5.5.2.