Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/andrews actions #67

Open
wants to merge 41 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
dd4af40
Adding github-download-release composite action.
andrewhaller Dec 1, 2022
c3fc59a
Adding discover-changed.
andrewhaller Dec 1, 2022
d27f23e
Adding terraform-compliance-aws.
andrewhaller Dec 1, 2022
231ffbc
Adding terragrunt-setup.
andrewhaller Dec 1, 2022
2c44c6b
Adding terragrunt-run-all.
andrewhaller Dec 1, 2022
955708f
Adding terragrunt-run-all file contents
andrewhaller Dec 1, 2022
3217f61
Adding aws-account-number.
andrewhaller Dec 1, 2022
5e2e781
Implementing some suggested changes.
andrewhaller Dec 1, 2022
bffe4ab
Commenting out token input to test passing env from parent.
andrewhaller Dec 1, 2022
a7fbc40
Implementing more generic terragrunt path input.
andrewhaller Dec 1, 2022
fc1f7d5
Using curl to download asset, instead of gh cli.
andrewhaller Dec 2, 2022
307a950
Changing output name.
andrewhaller Dec 2, 2022
a2ecc8f
Using GITHUB_API_URL.
andrewhaller Dec 6, 2022
2219fdd
Updating directory checks in terragrunt-setup action.
andrewhaller Dec 7, 2022
b2a032d
Checking for empty input in terragrunt-setup.
andrewhaller Dec 7, 2022
d1e7cdb
Using token input.
andrewhaller Dec 7, 2022
58b6219
Removing terragrunt-run-all.
andrewhaller Dec 8, 2022
dd1f4a6
Changing output name.
andrewhaller Dec 8, 2022
0763bf1
Updating description.
andrewhaller Dec 9, 2022
9b6abaa
Adding discover-filename.
andrewhaller Dec 9, 2022
6843b21
Updating terraform-compliance-aws.
andrewhaller Dec 9, 2022
a057622
Changed bash var name for consistency.
andrewhaller Dec 9, 2022
679681a
Defining referenced var.
andrewhaller Dec 9, 2022
fd8b982
Wrapping printf array value in quotes.
andrewhaller Dec 9, 2022
f27b4a1
Wrapping printf array value in quotes.
andrewhaller Dec 9, 2022
ffa9141
Adding git-config-pat.
andrewhaller Dec 9, 2022
7d2808d
Terraform compliance docker.
andrewhaller Dec 9, 2022
39a2bbc
Removing aws-account-number.
andrewhaller Dec 9, 2022
15d2084
Renaming terraform-compliance-aws to terraform-compliance-terragrunt.
andrewhaller Dec 9, 2022
adaf88d
Adding entrypoint contents.
andrewhaller Dec 9, 2022
a686e09
Fixed RANDOM issue.
andrewhaller Dec 9, 2022
f04c0b3
Updating default input.
andrewhaller Dec 9, 2022
d214063
Updating default input.
andrewhaller Dec 10, 2022
54409b0
Checking if file exists.
andrewhaller Dec 10, 2022
22b9ecd
Using mv instead of cp for performance.
andrewhaller Dec 10, 2022
723049f
Updating description.
andrewhaller Dec 10, 2022
c9936a8
Adding composite version of action.
andrewhaller Dec 10, 2022
1a50998
Action updates.
andrewhaller Dec 13, 2022
6bd29a3
Use openssl rand to get replacement key.
andrewhaller Dec 13, 2022
2ab8bd3
Updating actions.
andrewhaller Jan 6, 2023
f649fdd
Referencing action from branch instead of local.
andrewhaller Jan 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions aws-account-number/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: AWS Account Number
description: Retrieves an aws account number from a json file
inputs:
accounts-file:
description: A json file with the aws accounts
required: true
account-name:
description: The name of the specific account to use
required: true
account-name-label:
description: "The key name for the account name"
required: false
default: name
account-id-label:
description: "The key name for the account id"
required: false
default: account
outputs:
environments-to-run:
description: A list of environments that will have plan / apply executed on. This will be a JSON formatted array.
value: ${{ steps.account-number.outputs.number }}
runs:
using: composite
steps:
- name: Get AWS Account Number
shell: bash
id: account-number
run: |
output="$(cat ${{ inputs.accounts-file }} | jq -rc '.[] | select(.${{ inputs.account-name-label }} == "${{ inputs.account-name }}") | .${{ inputs.account-id-label }}')"
echo "number=$output"
echo "number=$output" >> $GITHUB_OUTPUT
38 changes: 38 additions & 0 deletions discover-changed/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Discover Changed Targets
description: Discovers a list of targets to execute plan/apply on
inputs:
targets:
description: A list of target folders to check when determining what to run plan/apply on. This should be a newline-delimited string of targets.
required: true
outputs:
changed-targets:
description: A list of targets that will have plan / apply executed on. This will be a JSON formatted array.
value: ${{ steps.set-matrix.outputs.matrix }}
runs:
using: composite
steps:
- name: Get Changed Files
id: files
uses: tj-actions/changed-files@v34
with:
files: ${{ inputs.targets }}

# given a list of files, output all of the targets that we need to plan against as JSON
# example output: ["liatrio-sandbox", "liatrio-non-prod"]
- name: List Changed Files
id: set-matrix
shell: bash
run: |
envs=()

files="${{ steps.files.outputs.all_modified_files }}"

if [ ! -z "$files" ]; then
for file in $files[@]; do
envs+=($(echo $file | cut -d'/' -f2))
done
fi

output="$(echo "${envs[@]}" | tr ' ' '\n' | sort -u | xargs echo -n | jq -R -s -c 'split(" ")')"
echo "matrix=${output}"
echo "matrix=${output}" >> $GITHUB_OUTPUT
62 changes: 62 additions & 0 deletions github-download-release/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
name: Github Download Release
description: Downloads the specified release using gh client
inputs:
token:
steve-liatrio marked this conversation as resolved.
Show resolved Hide resolved
description: The authorization token for the target repo
required: true
github-version:
description: Github version
steve-liatrio marked this conversation as resolved.
Show resolved Hide resolved
required: false
default: 'v2.0.0'
owner:
description: The repo owner
required: true
repo:
description: The repo name
required: true
release:
description: The release tag or number
required: true
format-ext:
description: The type of archive
required: false
default: linux_amd64.tar.gz
outputs:
filename:
description: The name of the downloaded tarball
value: ${{ steps.download-tarball.outputs.filename }}
runs:
using: composite
steps:
- name: Download github cli

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a reason to install the GH CLI at runtime? github-hosted runners should have it installed already

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah a github hosted runner would be idea. The repo being downloaded is this terraform-pipeline-executor project written in go. What it's doing under the hood requires the authentications in our self hosted runner.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah ok, so this is intended to be ran on self-hosted runners. A pattern I've seen for these types of actions is to check if the tool is already present, and only install if it's not present. If this action is only for use on self-hosted runners then maybe that's overkill, but it would be helpful to make it very obvious that the action shouldn't be used on gh-hosted runners. (maybe that's true of this entire repo and I just missed it?)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah they're pretty much all meant to be used in a situation where you have to use a self-hosted a runner. The discover-changed action is the only one that's used in a gh hosted runner. I could check for prior existence, however to be prudent, I would also need to check if the version of an already-present binary matches, as the user might want to override the version already installed. That said, for our specific case, if I had it my way, I would have control of the runner image and make sure the gh cli was on there to begin with. I think I'm landing on explaining the intent of use in the README

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could also add a skip-gh-if-already-present boolean option, whereby if it's present, and the user does not care about the version, then to skip the download of gh.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ended up doing away with gc cli altogether, and figured out the correct way to do a curl request on private repo.

shell: bash
env:
GH_VERSION: ${{ inputs.github-version }}
DOWNLOAD_FORMAT_EXT: ${{ inputs.format-ext }}
run: |
export GH_TAG=$(echo "${GH_VERSION}" | sed s/^v//g)
export DOWNLOAD_URL="https://github.com/cli/cli/releases/download/${GH_VERSION}/gh_${GH_TAG}_${DOWNLOAD_FORMAT_EXT}"
printf "Downloading '%s' ...\n" "${DOWNLOAD_URL}"
curl -LO "${DOWNLOAD_URL}"
export FILENAME="gh_${GH_TAG}_${DOWNLOAD_FORMAT_EXT}"
printf "Extracting '%s' ...\n" "${FILENAME}"
tar -xzvf "${FILENAME}"
export ARCH=$(echo $DOWNLOAD_FORMAT_EXT | awk -F'.' '{ print $1 }')
mv gh_${GH_TAG}_${ARCH}/bin/gh ./

- name: Download Tarball
shell: bash
id: download-tarball
env:
GITHUB_TOKEN: ${{ inputs.token }}
GITHUB_REPO: ${{ inputs.repo }}
GITHUB_REPO_OWNER: ${{ inputs.owner }}
RELEASE_NAME: ${{ inputs.release }}
DOWNLOAD_FORMAT_EXT: ${{ inputs.format-ext }}
run: |
export RELEASE_TAG=$(echo "${RELEASE_NAME}" | sed s/^v//g)
printf "Downloading release '%s' from repo '%s' ...\n" "${RELEASE_NAME}" "${GITHUB_REPO_OWNER}/${GITHUB_REPO}"
./gh release download --repo "${GITHUB_REPO_OWNER}/${GITHUB_REPO}" "${RELEASE_NAME}" -p "${GITHUB_REPO}_${RELEASE_TAG}_${DOWNLOAD_FORMAT_EXT}"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm lacking context on what this action is for, but this pattern (-p) seems very focused, while the action name implies it's fairly generic. Should the entire pattern be an input parameter?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The -p is "Download only assets that match a glob pattern" from this documentation https://cli.github.com/manual/gh_help_reference (search for "gh release download" on that page). This action is very specific in the sense that it downloads a archive of a public or private repo.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah ok, I missed that this was for downloading a repo archive. I assumed it was for downloading other artifacts from a release. Should the action name indicate that it's for that, specifically, then? Is there a reason to download the release archive of a repo instead of using git operations to pull a repo at a specific tag?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The repo being pulled is a private repo


output="${GITHUB_REPO}_${RELEASE_TAG}_${DOWNLOAD_FORMAT_EXT}"
echo "filename=${output}" >> $GITHUB_OUTPUT
56 changes: 56 additions & 0 deletions terraform-compliance-aws/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
name: AWS Terraform Compliance
description: Checks terraform complicance with aws
inputs:
target-folder:
description: The target folder
required: true
subfolders:
description: A space delimited list of sub-folders with terragrunt.hcl files
required: true
iam-role:
description: The iam role to assume
required: true
runs:
using: composite
steps:
- name: List Compliance folder
shell: bash
run: |
ls -la

- name: Terraform Compliance setup
shell: bash
env:
TARGET_FOLDER: ${{ inputs.target-folder }}
SUBFOLDERS: ${{ inputs.subfolders }}
TERRAGRUNT_IAM_ROLE: ${{ inputs.iam-role }}
run: |
cd $TARGET_FOLDER
steve-liatrio marked this conversation as resolved.
Show resolved Hide resolved
for i in $SUBFOLDERS
do
cd $i/
terragrunt plan -lock=false --out plan.out
terragrunt show -json plan.out > plan.out.json
cd ..
done

- name: Terraform Compliance
shell: bash
env:
TARGET_FOLDER: ${{ inputs.target-folder }}
SUBFOLDERS: ${{ inputs.subfolders }}
run: |
cd $TARGET_FOLDER
for i in $SUBFOLDERS
do
ii="${TARGET_FOLDER}/${i}"
echo "##################################################################################################################"
echo "TERRAFORM COMPLIANCE PATH ${ii}"
echo "##################################################################################################################"
cd $i/
terraform-compliance -n -p plan.out.json -f git:https://github.com/terraform-compliance/user-friendly-features.git
echo "##################################################################################################################"
echo "\n"
rm plan.out.json
cd ..
done
34 changes: 34 additions & 0 deletions terragrunt-run-all/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: Terragrunt Run All
description: Terragrunt run-all
inputs:
folder:
description: The target folder
required: true
iam-role:
description: The iam role to assume
required: true
bin-name:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are the path to and name of terragrunt parameterized? is it unsafe to assume terragrunt will be configured on PATH before the action is called?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I parameterized them to avoid assumptions that could not be overwritten. The default value for the terragrunt location is /usr/local/bin (and default name of binary being terragrunt). When used in conjuction with the terragrunt-setup action, the binary will be in the expected location so as not to require overwriting the defaults. However if terragrunt was downloaded or installed by another means, then the terragrunt location can be explicitly specified. Looking at it again, it does seem overly complicated to parameterize both values, and I could consolidate them into a single input, for example "terragrunt-bin-path".

description: The name of the executable
required: false
default: terragrunt
bin-path:
description: The bin location
required: false
default: /usr/local/bin
run-all-action:
description: The terragrunt command args
required: true
additional-options-args:
description: Additional terragrunt option args
required: false

runs:
using: composite
steps:

- name: Terrgrunt Apply
shell: bash
env:
TERRAGRUNT_IAM_ROLE: ${{ inputs.iam-role }}
run: |
${{ inputs.bin-path }}/${{ inputs.bin-name }} run-all ${{ inputs.run-all-action }} --terragrunt-non-interactive --terragrunt-working-dir ${{ inputs.folder }} ${{ inputs.additional-options-args }}
34 changes: 34 additions & 0 deletions terragrunt-setup/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: Terragrunt Setup
description: Checks terraform complicance with aws
inputs:
terragrunt-version:
description: The terragrunt version to use
required: false
default: 'v0.31.1'
bin-name:
description: The name of the executable
required: false
default: terragrunt
bin-dir:
description: The bin location
required: false
default: /usr/local/bin
format-ext:
description: The type of archive
required: false
default: linux_amd64
runs:
using: composite
steps:

- name: Terrgrunt Setup
shell: bash
env:
TERRAGRUNT_VERSION: ${{ inputs.terragrunt-version }}
DOWNLOAD_FORMAT_EXT: ${{ inputs.format-ext }}
run: |
export DOWNLOAD_URL="https://github.com/gruntwork-io/terragrunt/releases/download/${TERRAGRUNT_VERSION}/terragrunt_${DOWNLOAD_FORMAT_EXT}"
printf "Downloading '%s' ...\n" "${DOWNLOAD_URL}"
curl -f -Lo ${{ inputs.bin-name }} "${DOWNLOAD_URL}"
chmod +x ${{ inputs.bin-name }}
mv ${{ inputs.bin-name }} ${{ inputs.bin-path }}/${{ inputs.bin-name }}