Release Management

Branches

• main - production
• development - staging/testing
  • all feature branches merged into development
  • integration/QA testing on staging - doing on development branch
  • development branch merged into main
• security/hotfix branch - staging/testing
  • merge directly into main

Production release

• Weekly production release
  • with code/package/security updates
  • every tuesday - so we have time during the week if firefighting required
• Security/Hotfix production release
  • if dependabot or other security alert eg sast 'high' or 'critical' severity
  • severe bug in code eg scalability issues

Github workflow (yaml)

• docker-push-<operator>
  • NOTE: Select the required branch from Github Actions tab before triggering manual workflows for pushing docker staging images to dockerhub
• deploy-staging
  • deploy docker containers to staging with dev requirements
• deploy-production
  • deploy docker containers to production without dev requirements
  • notify on slack
• On PR - development or security/hotfix branch
  • sast security testing - high+ severity eg bandit
  • all unit tests running

Note: this is not technically possible since it is a security risk. Manually do the following steps
Refer: 
- https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions#using-secrets-in-a-workflow
- https://securitylab.github.com/research/github-actions-preventing-pwn-requests/

• ON merge - development or security/hotfix

  • deploy latest:dev operators to dockerhub
  • deploy-staging

• ON Merge - main

  • create versioned release/tag with latest PR commit on github
  • deploy versioned:production operators to dockerhub
  • deploy-production
    • notify on slack
    • we should also setup locust production monitoring for failure event thresholds
  • sync development and security/hotfix branch with main

• ON Adding new operator

  • update .ci/docker-compose.yml - add tests to run in ci
  • update pr-security.yml workflow - add pip audit for operator requirements

• ON Adding new worker (operator)

  • update merge-main.yml workflow - push operator images to dockerhub
  • update pyproject.toml - add assets new operator to release