Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add project template #65

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

add project template #65

wants to merge 10 commits into from

Conversation

philvarner
Copy link

@philvarner philvarner commented May 23, 2024

Related issue(s)

  • n/a

Proposed Changes

  1. Add deploy project template to this repo, with detailed usage instructions.

Testing

This change was validated by the following observations:

  1. n/a

Checklist

  • I have deployed and validated this change
  • Changelog
    • I have added my changes to the changelog
    • No changelog entry is necessary
  • README migration
    • I have added any migration steps to the Readme
    • No migration is necessary

@philvarner philvarner marked this pull request as ready for review May 28, 2024 15:45
- Effect: Allow
Principal:
AWS:
- "arn:aws:iam::433175837143:root"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- "arn:aws:iam::433175837143:root"
- !Join
- ":"
-
- "arn:aws:iam:"
- !Ref GitLabRunnerAWSAccountNumber
- "root"

Default: appFilmDropDeployRole
GitLabRolePolicyName:
Type: String
Default: appFilmDropDeployPolicy
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Default: appFilmDropDeployPolicy
Default: appFilmDropDeployPolicy
GitLabRunnerAWSAccountNumber:
Type: String

Comment on lines +86 to +97
You can create a CloudFormation Stack from these templates with the
following command (GitLab example shown), The Terraform state S3 Bucket name
(discussed later) must be globally unique, so it is recommended to use where the value
passed for `TerraformStateBucketName` is a unique bucket name based on the template
`filmdrop-{project_name}-{region}-terraform-state-{random_string}`:

```shell
aws cloudformation deploy --stack-name "appFilmDropDeployRoleBootstrap" \
--template-file bootstrap/gitlab_deploy_role_cfn.yml \
--capabilities=CAPABILITY_NAMED_IAM \
--parameter-overrides TerraformStateBucketName=filmdrop-stingray-us-west-2-terraform-state-20240515
```
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
You can create a CloudFormation Stack from these templates with the
following command (GitLab example shown), The Terraform state S3 Bucket name
(discussed later) must be globally unique, so it is recommended to use where the value
passed for `TerraformStateBucketName` is a unique bucket name based on the template
`filmdrop-{project_name}-{region}-terraform-state-{random_string}`:
```shell
aws cloudformation deploy --stack-name "appFilmDropDeployRoleBootstrap" \
--template-file bootstrap/gitlab_deploy_role_cfn.yml \
--capabilities=CAPABILITY_NAMED_IAM \
--parameter-overrides TerraformStateBucketName=filmdrop-stingray-us-west-2-terraform-state-20240515
```
You can create a CloudFormation Stack from these templates with the
following command (GitLab example shown), the GitLabRunner AWS Account Number
is needed to authorize the GitLab runner to assume the appFilmDropDeployRole:
```shell
aws cloudformation deploy --stack-name "appFilmDropDeployRoleBootstrap" \
--template-file bootstrap/gitlab_deploy_role_cfn.yml \
--capabilities=CAPABILITY_NAMED_IAM \
--parameter-overrides GitLabRunnerAWSAccountNumber=123456789123

Comment on lines +132 to +135
Terraform stores the state of the deployment in a managed state file. When deploying via CI
to shared environments, this state file must be stored somewhere non-local, so we use
S3. This sets up an S3+DynamoDB Terraform backend to store this state. This
should be created once per AWS Account + Region.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Terraform stores the state of the deployment in a managed state file. When deploying via CI
to shared environments, this state file must be stored somewhere non-local, so we use
S3. This sets up an S3+DynamoDB Terraform backend to store this state. This
should be created once per AWS Account + Region.
Terraform stores the state of the deployment in a managed state file. When deploying via CI
to shared environments, this state file must be stored somewhere non-local, so we use
S3. This sets up an S3+DynamoDB Terraform backend to store this state. This
should be created once per AWS Account + Region. The Terraform state S3 Bucket name
must be globally unique, so it is recommended to use where the value
passed for `TerraformStateBucketName` is a unique bucket name based on the template
`filmdrop-{project_name}-{region}-terraform-state-{random_string}`:

Comment on lines +121 to +128
```shell
cd bootstrap
terraform init
terraform apply -target=aws_iam_service_linked_role.opensearch_linked_role
```

There is no need to keep the terraform state file, as running this is intended to
be an idempotent operation.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could just aws iam create-service-linked-role --aws-service-name opensearchservice.amazonaws.com?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found I also had to aws iam create-service-linked-role --aws-service-name wafv2.amazonaws.com to make something using the WAF happy.

## Pre-deploy setup

1. For each of the AWS Accounts to be deployed into, create the bootstrap
resources as outlined in <bootstrap/README.md>.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file doesn't seem to exist. I worked out that the info was in the deploy.md file in the repo root, but it threw me when I couldn't find bootstrap instructions in the copied project.

Comment on lines +68 to +80
- export STAC_SERVER_DIR="stac-server-${STAC_SERVER_TAG:1}"
- source $HOME/.nvm/nvm.sh
- nvm use v18
- curl -L -f --no-progress-meter -o - "https://github.com/stac-utils/stac-server/archive/refs/tags/${STAC_SERVER_TAG}.tar.gz" | tar -xz
- cd "$STAC_SERVER_DIR"
- npm install
- BUILD_PRE_HOOK=true npm run build
- mkdir -p $FILMDROP_BUILD_DIR/modules/stac-server/lambda/api
- cp dist/api/api.zip $FILMDROP_BUILD_DIR/modules/stac-server/lambda/api/
- mkdir -p $FILMDROP_BUILD_DIR/modules/stac-server/lambda/ingest
- cp dist/ingest/ingest.zip $FILMDROP_BUILD_DIR/modules/stac-server/lambda/ingest/
- mkdir -p $FILMDROP_BUILD_DIR/modules/stac-server/lambda/pre-hook
- cp dist/pre-hook/pre-hook.zip $FILMDROP_BUILD_DIR/modules/stac-server/lambda/pre-hook/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aren't the stac-server lambda zips included in the terraform module, and thus this build step is just duplication? I'd argue we shouldn't be including the zips in the terraform module source, but also think we should have a build step that does pull these down into a build artifact of the whole module so users don't have to worry about any of this. Especially as we add more lambdas to build, i.e., cirrus.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants