Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

By default Ice fails to use S3 on eu-central-1 #215

Open
multani opened this issue May 30, 2016 · 1 comment · May be fixed by #262
Open

By default Ice fails to use S3 on eu-central-1 #215

multani opened this issue May 30, 2016 · 1 comment · May be fixed by #262

Comments

@multani
Copy link
Contributor

multani commented May 30, 2016

I configured all the things in AWS as explained in the README of Ice and as we are mostly (99%) using AWS from eu-central-1, I setup the S3 bucket to hold the billing reports on this datacenter.

After starting up and download the instances prices and everything, it successfully manages to download some things from this bucket on eu-central-1 but eventually fails with the following error:

ice_1  | 2016-05-30 13:41:04,902 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor  - trying to download 123456789-aws-billing-detailed-line-items-2016-04.csv.zip...
ice_1  | | Error 2016-05-30 13:41:05,068 [com.netflix.ice.processor.BillingFileProcessor] ERROR processor.BillingFileProcessor  - Error polling
ice_1  | Message: Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: XXXXXXXXXXXXX)
ice_1  |     Line | Method
ice_1  | ->> 1077 | handleErrorResponse        in com.amazonaws.http.AmazonHttpClient
ice_1  | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
ice_1  | |    725 | executeOneRequest          in     ''
ice_1  | |    460 | executeHelper . . . . . .  in     ''
ice_1  | |    295 | execute                    in     ''
ice_1  | |   3710 | invoke . . . . . . . . . . in com.amazonaws.services.s3.AmazonS3Client
ice_1  | |   1010 | getObjectMetadata          in     ''
ice_1  | |    988 | getObjectMetadata . . . .  in     ''
ice_1  | |    275 | downloadFileIfChangedSince in com.netflix.ice.common.AwsUtils
ice_1  | |    175 | poll . . . . . . . . . . . in com.netflix.ice.processor.BillingFileProcessor
ice_1  | |     50 | doWork                     in com.netflix.ice.common.Poller
ice_1  | |     28 | access$000 . . . . . . . . in     ''
ice_1  | |     88 | run                        in com.netflix.ice.common.Poller$1
ice_1  | ^    745 | run . . . . . . . . . . .  in java.lang.Thread

(running from Docker Compose)

It was hard to find more information, but I managed to upgrade the log levels, saw the actual HTTP HEAD query made and tried to do the same with a GET instead and I got the following error message instead:

<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>InvalidRequest</Code>
<Message>The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.</Message>
</Error>

For some reason, this mechanism is not used directly and I guess that's why it fails on eu-central-1 which doesn't support other mechanism (as far as I understand).

It looks like #193 is here to address this problem although both the explanation and the fix are confusing.

After looking more closely at the code, it appears that if I set the EC2_REGION system property to eu-central-1 at startup it solves the problem, but:

  1. This is not documented anywhere;
  2. I'm pretty sure there's a way to get the authentication mechanism right, even it the user doesn't manually specify the S3 region to use (or that's what the AWS SDK promise seems to be?)

It would be cool if at least the first point could be addressed so users using other regions than the US ones are also able to start Ice, although I'm not sure exactly why this setting was introduced in the first place.

Probably a better fix would be to use the GetLocation method on the bucket to get where the location of the bucket is and set the correct endpoint then.
@snakelab
Copy link

snakelab commented Jun 1, 2017 •
edited

hi, thanks for this information - but this does not fix my installation :-/

#!/bin/bash

AWS_PROFILE=dev
AWS_DEFAULT_REGION=eu-central-1
EC2_REGION=eu-central-1
docker-compose -f ./docker-compose.yml up

@multani multani mentioned this issue Jul 26, 2017
Open
@aantonw aantonw linked a pull request Aug 5, 2017 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add region poperty for working bucket aantonw/ice
2 participants
@multani @snakelab