Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add eks cluster gha #112

Merged
merged 52 commits into from
Sep 18, 2024
Merged
Show file tree
Hide file tree
Changes from 38 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
9aef1b3
feat: add eks cluster gha
leiicamundi Sep 11, 2024
afd8508
don't install if already installed
leiicamundi Sep 11, 2024
5cee55c
add some params
leiicamundi Sep 11, 2024
794cfe8
copy missing backend
leiicamundi Sep 11, 2024
df35114
add missing parameters
leiicamundi Sep 11, 2024
1bcf4e3
fix param
leiicamundi Sep 11, 2024
65ae75a
fix param
leiicamundi Sep 11, 2024
a99a8fb
add tmate debug
leiicamundi Sep 12, 2024
8e35b9e
tmp disable tests
leiicamundi Sep 12, 2024
9e35eba
detach
leiicamundi Sep 12, 2024
56838a7
fix
leiicamundi Sep 12, 2024
b31cb08
fix delete cluster
leiicamundi Sep 12, 2024
11b0485
add utility acton
leiicamundi Sep 12, 2024
5de0684
fix missing action
leiicamundi Sep 12, 2024
921c4e7
fix action
leiicamundi Sep 12, 2024
8746379
add aurora in the tests
leiicamundi Sep 12, 2024
e0954f5
cidrs and vpc is now optional for aurora
leiicamundi Sep 12, 2024
eb874ad
fix inded
leiicamundi Sep 12, 2024
67eebe7
add default cidrs
leiicamundi Sep 12, 2024
7b6c58a
fix tags
leiicamundi Sep 12, 2024
607c01d
remove tags
leiicamundi Sep 12, 2024
fc8ba7d
default vpc
leiicamundi Sep 12, 2024
f8228bf
default vpc
leiicamundi Sep 12, 2024
0f7dba7
fix
leiicamundi Sep 12, 2024
67fd689
subnet is now optionnal
leiicamundi Sep 12, 2024
0244eb2
integrate aurora gha in eks
leiicamundi Sep 13, 2024
9d329b9
fix index
leiicamundi Sep 13, 2024
e9e6805
ensure output is single line
leiicamundi Sep 13, 2024
00133d8
fix tee
leiicamundi Sep 13, 2024
9e39163
fix values
leiicamundi Sep 13, 2024
3318c31
fix types
leiicamundi Sep 13, 2024
3290687
fix inputs
leiicamundi Sep 13, 2024
2182d64
fix availability zones
leiicamundi Sep 13, 2024
6e9f87c
update action
leiicamundi Sep 13, 2024
f80203e
fix iam roles
leiicamundi Sep 13, 2024
177760a
enforce list of strings
leiicamundi Sep 13, 2024
214d0f3
pin versions and cleanup
leiicamundi Sep 13, 2024
bfff6a3
fix params
leiicamundi Sep 16, 2024
5f40c90
realign doc of gha and params
leiicamundi Sep 16, 2024
633c098
reformat yaml using yamlftm
leiicamundi Sep 16, 2024
ece9e08
use tf-state-key
leiicamundi Sep 16, 2024
cf649ff
auto generate action readmes
leiicamundi Sep 16, 2024
db88d39
use additional variables for non default on eks
leiicamundi Sep 16, 2024
8db575c
remove outdate parameter
leiicamundi Sep 16, 2024
568a642
update pre-commits
leiicamundi Sep 16, 2024
c73b2d9
fix user mapping
leiicamundi Sep 16, 2024
d84fb16
Merge branch 'main' into feature/gha-eks
leiicamundi Sep 16, 2024
8415dae
fix var file order and action generation
leiicamundi Sep 17, 2024
5d961bc
fix inputs tfvars
leiicamundi Sep 17, 2024
07478b3
fix plan file
leiicamundi Sep 17, 2024
737704d
fix deployment
leiicamundi Sep 17, 2024
102008e
use new generation of docs
leiicamundi Sep 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 87 additions & 0 deletions .github/actions/aurora-manage-cluster/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# Deploy RDS Aurora Cluster GitHub Action

This GitHub Action automates the deployment of an Amazon RDS Aurora cluster using Terraform. It installs Terraform and AWS CLI, and outputs the Aurora cluster endpoint along with other relevant details.

## Description

The **Deploy RDS Aurora Cluster** action enables you to:

- Automate the deployment of an RDS Aurora cluster on AWS.
- Use Terraform for infrastructure as code.
- Install specific versions of Terraform and AWS CLI.
- Output the Aurora cluster endpoint, Terraform state URL, and all other Terraform outputs dynamically.

## Inputs

The following inputs are required or optional for the action:

| Input | Description | Required | Default |
leiicamundi marked this conversation as resolved.
Show resolved Hide resolved
|-------|-------------|----------|---------|
| `cluster-name` | Name of the RDS Aurora cluster to deploy. | Yes | - |
| `engine-version` | Version of the Aurora engine to use. | Yes | see `action.yml` |
| `instance-class` | Instance class for the Aurora cluster. | Yes | `db.t3.medium` |
| `num-instances` | Number of instances in the Aurora cluster. | Yes | `1` |
| `username` | Username for the PostgreSQL admin user. | Yes | - |
| `password` | Password for the PostgreSQL admin user. | Yes | - |
| `vpc-id` | VPC ID to create the cluster in. | Yes | - |
| `subnet-ids` | List of subnet IDs to create the cluster in. | Yes | - |
| `cidr-blocks` | CIDR blocks to allow access from and to. | Yes | - |
| `auto-minor-version-upgrade` | If true, minor engine upgrades will be applied automatically to the DB instance during the maintenance window. | No | `true` |
| `availability-zones` | Array of availability zones to use for the Aurora cluster. | No | `[]` |
| `iam-roles` | Allows propagating additional IAM roles to the Aurora cluster for features like access to S3. | No | `[]` |
| `iam-auth-enabled` | Determines whether IAM authentication should be activated for IRSA usage. | No | `false` |
| `ca-cert-identifier` | Specifies the identifier of the CA certificate for the DB instance. | No | `rds-ca-rsa2048-g1` |
| `default-database-name` | The name for the automatically created database on cluster creation. | No | `camunda` |
| `s3-backend-bucket` | Name of the S3 bucket to store Terraform state. | Yes | - |
| `s3-bucket-region` | Region of the bucket containing the resources states. Fallbacks to `aws-region` if not set. | No | - |
| `tf-modules-revision` | Git revision of the Terraform modules to use. | Yes | `main` |
| `tf-modules-path` | Path where the Terraform Aurora modules will be cloned. | Yes | `./.action-tf-modules/aurora/` |
| `tf-cli-config-credentials-hostname` | The hostname of a HCP Terraform/Terraform Enterprise instance for the CLI configuration file. | No | `app.terraform.io` |
| `tf-cli-config-credentials-token` | The API token for a HCP Terraform/Terraform Enterprise instance. | No | - |
| `tf-terraform-version` | The version of Terraform CLI to install. | No | `latest` |
| `tf-terraform-wrapper` | Whether to install a wrapper for the Terraform binary. | No | `true` |
| `awscli-version` | Version of the AWS CLI to use. | Yes | see `action.yml` |

## Outputs

The action provides the following outputs:

| Output | Description |
|--------|-------------|
| `aurora-endpoint` | The endpoint of the deployed Aurora cluster. |
| `terraform-state-url` | URL of the Terraform state file in the S3 bucket. |
| `all-terraform-outputs` | All outputs from Terraform. |

## Usage

To use this GitHub Action, include it in your workflow file:

```yaml
jobs:
deploy_aurora:
runs-on: ubuntu-latest
steps:
- name: Deploy Aurora Cluster
uses: camunda/camunda-tf-eks-module/aurora-manage-cluster@main
with:
cluster-name: 'my-aurora-cluster'
engine-version: '15.4'
instance-class: 'db.t3.medium'
num-instances: '2'
username: 'admin'
password: ${{ secrets.DB_PASSWORD }}
vpc-id: 'vpc-12345678'
subnet-ids: 'subnet-12345,subnet-67890'
cidr-blocks: '10.0.0.0/16'
auto-minor-version-upgrade: 'true'
availability-zones: '["us-west-2a", "us-west-2b"]'
iam-roles: '["arn:aws:iam::123456789012:role/my-role"]'
iam-auth-enabled: 'false'
ca-cert-identifier: 'rds-ca-rsa2048-g1'
default-database-name: 'mydatabase'
s3-backend-bucket: 'my-terraform-state-bucket'
s3-bucket-region: 'us-west-2'
tf-modules-revision: 'main'
tf-modules-path: './.action-tf-modules/aurora/'
awscli-version: '2.15.52'
```
202 changes: 202 additions & 0 deletions .github/actions/aurora-manage-cluster/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,202 @@
name: Deploy RDS Aurora Cluster

description: |
This GitHub Action automates the deployment of an RDS Aurora cluster using Terraform.
This action will also install Terraform and awscli. It will output the Aurora cluster endpoint.

inputs:
cluster-name:
description: 'Name of the RDS Aurora cluster to deploy'
required: true

engine-version:
description: 'Version of the Aurora engine to use'
required: true
# renovate: datasource=endoflife-date depName=amazon-rds-postgresql versioning=semver
default: "15.4"
leiicamundi marked this conversation as resolved.
Show resolved Hide resolved

instance-class:
description: 'Instance class for the Aurora cluster'
required: true
default: "db.t3.medium"
leiicamundi marked this conversation as resolved.
Show resolved Hide resolved

num-instances:
description: 'Number of instances in the Aurora cluster'
required: true
default: "1"

username:
description: 'Username for the PostgreSQL admin user'
required: true

password:
description: 'Password for the PostgreSQL admin user'
required: true

vpc-id:
description: 'VPC ID to create the cluster in'
required: true

subnet-ids:
description: 'List of subnet IDs to create the cluster in'
required: true

cidr-blocks:
description: 'CIDR blocks to allow access from and to'
required: true

auto-minor-version-upgrade:
description: 'If true, minor engine upgrades will be applied automatically to the DB instance during the maintenance window'
default: "true" # Default value from variable.tf

availability-zones:
description: 'Array of availability zones to use for the Aurora cluster'
default: ''
leiicamundi marked this conversation as resolved.
Show resolved Hide resolved

iam-roles:
description: 'Allows propagating additional IAM roles to the Aurora cluster to allow e.g. access to S3'
default: "[]" # Default value from variable.tf

iam-auth-enabled:
description: 'Determines whether IAM auth should be activated for IRSA usage'
default: "false" # Default value from variable.tf

ca-cert-identifier:
description: 'Specifies the identifier of the CA certificate for the DB instance'
default: 'rds-ca-rsa2048-g1' # Default value from variable.tf

default-database-name:
description: 'The name for the automatically created database on cluster creation.'
default: 'camunda' # Default value from variable.tf

s3-backend-bucket:
description: 'Name of the S3 bucket to store Terraform state'
required: true
s3-bucket-region:
description: 'Region of the bucket containing the resources states'
tf-modules-revision:
description: 'Git revision of the tf modules to use'
default: 'main'
required: true

tf-modules-path:
description: 'Path where the tf Aurora modules will be cloned'
default: './.action-tf-modules/aurora/'
required: true

# inherited from https://github.com/hashicorp/setup-terraform/blob/main/action.yml
tf-cli-config-credentials-hostname:
description: 'The hostname of a HCP Terraform/Terraform Enterprise instance to place within the credentials block of the Terraform CLI configuration file. Defaults to `app.terraform.io`.'
default: 'app.terraform.io'
required: false
tf-cli-config-credentials-token:
description: 'The API token for a HCP Terraform/Terraform Enterprise instance to place within the credentials block of the Terraform CLI configuration file.'
required: false
tf-terraform-version:
description: 'The version of Terraform CLI to install. Defaults to `latest`.'
default: 'latest'
required: false
tf-terraform-wrapper:
description: 'Whether or not to install a wrapper to wrap subsequent calls of the `terraform` binary and expose its STDOUT, STDERR, and exit code as outputs named `stdout`, `stderr`, and `exitcode` respectively. Defaults to `true`.'
default: 'true'
required: false
awscli-version:
description: 'Version of the aws cli to use'
required: true
# renovate: datasource=github-releases depName=aws/aws-cli
default: "2.15.52"

outputs:
aurora-endpoint:
description: 'The endpoint of the deployed Aurora cluster'
value: ${{ steps.apply.outputs.aurora_endpoint }}

terraform-state-url:
description: 'URL of the Terraform state file in the S3 bucket'
value: ${{ steps.utility.outputs.terraform-state-url }}

# Add all terraform outputs dynamically
all-terraform-outputs:
description: 'All outputs from Terraform'
value: ${{ steps.fetch_outputs.outputs.all_terraform_outputs }}

runs:
using: 'composite'
steps:
- name: Use Utility Actions
id: utility
# seehttps://github.com/orgs/community/discussions/41927 it's not possible to optimize this yet
# steps.uses cannot access the github context.
# uses: ${{ github.action_repository }}/utility-action@${{ github.action_ref }}
# TODO: hard pin this one once it's merged
#uses: "camunda/camunda-tf-eks-module/utility-action@main"
uses: "./.github/actions/utility-action"
leiicamundi marked this conversation as resolved.
Show resolved Hide resolved
with:
awscli-version: '${{ inputs.awscli-version }}'
terraform-version: '${{ inputs.terraform-version }}'

s3-backend-bucket: '${{ inputs.s3-backend-bucket }}'
s3-bucket-region: '${{ inputs.s3-bucket-region }}'

name: '${{ inputs.cluster-name }}'
module-name: 'aurora'

tf-cli-config-credentials-hostname: '${{ inputs.tf-cli-config-credentials-hostname }}'
tf-cli-config-credentials-token: '${{ inputs.tf-cli-config-credentials-token }}'
tf-terraform-wrapper: '${{ inputs.tf-terraform-wrapper }}'

- name: Checkout Repository Aurora modules
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4
with:
repository: "camunda/camunda-tf-eks-module"
ref: ${{ inputs.tf-modules-revision }}
path: ${{ inputs.tf-modules-path }}
fetch-depth: 0

- name: Terraform Init
shell: bash
id: init
working-directory: "${{ inputs.tf-modules-path }}/modules/aurora/"
run: |
cp ../fixtures/backend.tf ./
terraform version
terraform init -backend-config="bucket=${{ steps.utility.outputs.TFSTATE_BUCKET }}" -backend-config="key=${{ steps.utility.outputs.TFSTATE_KEY }}" -backend-config="region=${{ steps.utility.outputs.TFSTATE_REGION }}"
terraform validate -no-color

- name: Terraform Plan
shell: bash
id: plan
working-directory: "${{ inputs.tf-modules-path }}/modules/aurora/"
run: |
terraform plan -no-color -out aurora.plan -var "cluster_name=${{ inputs.cluster-name }}" \
-var "engine_version=${{ inputs.engine-version }}" \
-var "instance_class=${{ inputs.instance-class }}" \
-var "num_instances=${{ inputs.num-instances }}" \
-var "username=${{ inputs.username }}" \
-var "password=${{ inputs.password }}" \
-var "auto_minor_version_upgrade=${{ inputs.auto-minor-version-upgrade }}" \
-var 'availability_zones=${{ inputs.availability-zones }}' \
-var 'iam_roles=${{ inputs.iam-roles }}' \
-var "iam_auth_enabled=${{ inputs.iam-auth-enabled }}" \
-var "ca_cert_identifier=${{ inputs.ca-cert-identifier }}" \
-var "default_database_name=${{ inputs.default-database-name }}" \
-var "vpc_id=${{ inputs.vpc-id }}" \
-var 'subnet_ids=${{ inputs.subnet-ids }}' \
-var 'cidr_blocks=${{ inputs.cidr-blocks }}'

- name: Terraform Apply
shell: bash
id: apply
working-directory: "${{ inputs.tf-modules-path }}/modules/aurora/"
run: |
terraform apply -no-color aurora.plan
export aurora_endpoint="$(terraform output -raw aurora_endpoint)"
echo "aurora_endpoint=$aurora_endpoint" >> "$GITHUB_OUTPUT"

- name: Fetch Terraform Outputs
shell: bash
id: fetch_outputs
working-directory: "${{ inputs.tf-modules-path }}/modules/aurora/"
run: |
all_outputs=$(terraform output -json | jq -c .)
echo "all_terraform_outputs=$all_outputs" | tee -a "$GITHUB_OUTPUT"
leiicamundi marked this conversation as resolved.
Show resolved Hide resolved
39 changes: 39 additions & 0 deletions .github/actions/eks-cleanup-resources/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Delete EKS Resources

This GitHub Action automates the deletion of AWS resources using a shell script. It helps you manage and clean up modules of this repository as resources by specifying a target or deleting resources based on age criteria.

## Usage

To use this action, include it in your workflow file (e.g., `.github/workflows/delete-eks-resources.yml`):

```yaml
name: Delete EKS Resources

on:
workflow_dispatch:

jobs:
cleanup:
runs-on: ubuntu-latest
steps:
- name: Delete EKS resources
uses: camunda/camunda-tf-eks-module/eks-cleanup-resources@main
with:
tf-bucket: 'your-s3-bucket-name'
tf-bucket-region: 'your-region'
max-age-hours: 24
target: 'all'
temp-dir: './tmp/eks-cleanup/'
```

## Inputs

The action supports the following input parameters:

| Input Name | Description | Required | Default |
|--------------------|-------------------------------------------------------------------------------------------|----------|----------------------------|
| `tf-bucket` | The S3 bucket containing the resources' state files. | Yes | N/A |
| `tf-bucket-region` | The region of the S3 bucket containing the resources state files. Falls back to `AWS_REGION` if not set. | No | AWS_REGION |
| `max-age-hours` | The maximum age (in hours) for resources to be deleted. | No | "20" |
| `target` | Specifies an ID to destroy specific resources or "all" to destroy all resources. | Yes | "all" |
leiicamundi marked this conversation as resolved.
Show resolved Hide resolved
| `temp-dir` | Temporary directory prefix used for storing resource data during processing. | No | "./tmp/eks-cleanup/" |
Loading
Loading