Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to set custom name for CUR bucket #956

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

rclemente-mrge
Copy link

@rclemente-mrge rclemente-mrge commented Oct 1, 2024

Issue #, if available:

Description of changes:

Ability to use a different bucket name instead of the default format

@iakov-aws
Copy link
Collaborator

@sean-nixon can you check this please?

Here we adjust source bucket but not replication one?

@rclemente-mrge
Copy link
Author

rclemente-mrge commented Oct 1, 2024

@iakov-aws
Adding some context.

Currently our company already has CUR in a bucket and we want to import that into this module for creating the Cudos board

But we don't have a replication bucket yet.

@iakov-aws
Copy link
Collaborator

thanks for the context

We do not fully support this usecase and do not recommend using existing CUR.
https://catalog.workshops.aws/awscid/en-US/faqs#can-i-use-an-existing-cost-and-usage-report-(cur)-instead-of-the-one-created-by-cid

The main reason being that different customers can use different settings of CUR creating tens of combinations that we do not support.

Many use 3rd party tools that have their own requirements for CUR format and content and 9 out 10 are incompatible with others (and between tools as well).

Also there is a very special prefix convention allowing CID to collect data across multiple AWS Organization and customer's cur surely do not have this.

So the best way is to create a new CUR and backfill data for 3 years.

@iakov-aws
Copy link
Collaborator

Typically the cost of s3 is low even for huge customers and having many CURs is not any issue.

@iakov-aws
Copy link
Collaborator

One valid reason for this change would be to use customer's specific naming convention.

@CorinneVerheyde
Copy link

I would be interessed too by the ability to use/specify an existing bucket.

  1. We are working on an automatic deployment of the whole CID solution. During the testing process, we have to deploy/destroy several time and it's a shame to have to empty the cur bucket each time to destroy it, then to recreate it 5 minutes later.

  2. Use customer's specific naming convention, and be able to use different name for source or destination cur. For this, we would need both source bucket and replication one to be customized.

@iakov-aws
Copy link
Collaborator

I see. This is an interesting use-case. Totally see value of it.

Currently in CUR/DataExports we operate with bucket replication mechanisms that require replication rules and bucket policies/

  • In destination account Bucket policies theoretically can be installed separately and this will only will work if bucket has no policy at all.
  • In source account Bucket replication in source cannot be done outside of bucket definition at all.

We can try to implement this for destination account only.
Today the easiest would be to sync the data to a backup bucket and to sync it back once the stack is updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants