Support internal load balancers

[3u13r]

Context

Proposed change(s)

• Each CSP terraform has a internal_loadbalancer flag which default can be switched to true. Then the constellation infra is set up using internal LB(s) only.
• Remove the lb svc for the verify deployment

I manually verified this on every of the 3 CSPs.

Should we build tests for this before merging?
If yes, how?
One way could be to automatically create a single VM with public IP and SSH access in the right subnet when var.internal_loadbalancer && var.debug is true. We'd need to write that code in our current terraform tough.

How to (manually) test? (old)

• Set the default of internal_loadbalancer to true in the var.tf file of the CSP.
• run your devbuild
• run constellation create as normal
• create a VM with public IP and the necessary SSH access in the subnet of the loadbalancer
• scp -r <path to workdir> user@publicIP:~
• ssh into the machine
• Disable the license check
• run cdbg deploy and constellation init as normal

Note that on gcp and aws you need more cloud permission inside the VM since you are not logged in with your user.
The simplest ways are:

• AWS: during VM creation (or a stopped VM) add the worker node IAM role to the VM
• GCP: during VM create allow read access to the "Compute" API

How to (manually) test?

• create a constellation with internalLoadBalancer: true and debug: true in your constellation-conf.yaml. Then deploy and initialize your Constellation as usual. See the jump-host VM in the CSP.

Note that on Azure one needs to disable konnectivity inside the bootstrapper like so: 4a6b99d. Since we will drop konnectivity after the Cilium update, I decided to not build another workaround for it. The underlying problem is the same as described above the PrepareControlPlane function in the azure package. Only with the added problem that the exact same solution as done in the function does not work, since konnectivity is a pod so the routing is handled by Cilium.

Building testing debug image: https://github.com/edgelesssys/constellation/actions/runs/6494433304
ref/feat-arch-internal-lb/stream/debug/v2.13.0-pre.0.20231012022300-8d7e3b676dcd

E2E Test GCP: https://github.com/edgelesssys/constellation/actions/runs/6419796210/job/17528606182

Checklist

• Update docs
• Add labels (e.g., for changelog category)
• Is PR title adequate for changelog?
• Link to Milestone

[netlify]

✅ Deploy Preview for constellation-docs ready!

Name	Link
🔨 Latest commit	e924f23
🔍 Latest deploy log	https://app.netlify.com/sites/constellation-docs/deploys/652e8de2ca838a0008bb74db
😎 Deploy Preview	https://deploy-preview-2388--constellation-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[3u13r]

I discussed with @malt3 a possible testing which should be compatible with most (all?) of our current CI tests.

Essentially:

1. create a jump host VM with public IP which runs socat.
2. run constellation create
3. hack the PublicIP into the constellation-id.json as customEndpoint so that the kubeAPI adds it as SAN.
4. change the ClusterEndpoint to the PublicIP
5. run constellation init
6. replace the endpoint in the kubeconfig with the PublicIP

Now we should be able to operate as normal from our local system / the CI.

Thanks @malt3 for the discussion!

[daniel-weisse]

Why have this as a separate test instead of making it an option for the e2e-test-manual workflow?

[3u13r]

Because the e2e-test-manual has already 10 dispatch event intputs and I was uncomfortable merging even more inputs.

[elchead]

I looked at the Terraform changes which lgtm.

I discussed with @malt3 a possible testing which should be compatible with most (all?) of our current CI tests.

Essentially:

1. create a jump host VM with public IP which runs socat.
2. run constellation create
3. hack the PublicIP into the constellation-id.json as customEndpoint so that the kubeAPI adds it as SAN.
4. change the ClusterEndpoint to the PublicIP
5. run constellation init
6. replace the endpoint in the kubeconfig with the PublicIP

Now we should be able to operate as normal from our local system / the CI.

Thanks @malt3 for the discussion!

Thanks for the test coverage! Should we run them as part of the weekly (at least for GCP)?

[3u13r]

The commit add in-cluster endpoint to terraform output is necessary to add the in-cluster endpoint to the Cilium helm values.

[derpsteb]

Only looked at Helm pkg. 👍

[github-actions]

Coverage report

Package	Old	New	Trend
bootstrapper/cmd/bootstrapper	[no test files]	[no test files]	🚧
cli/internal/cloudcmd	63.90%	63.90%	↔️
cli/internal/helm	49.50%	49.50%	🚧
cli/internal/state	94.80%	95.00%	↗️
cli/internal/terraform	71.30%	70.90%	↘️
internal/cloud/aws	82.60%	84.40%	↗️
internal/cloud/azure	71.40%	67.10%	↘️
internal/config	80.20%	80.20%	↔️