use upstream coredns chart instead of fork

[abaguas]

CoreDNS is a core component of the K8GB application. Until now we were installing it using a fork of the official helm chart with the following diff. The fork was necessary because Kubernetes did not support services of type load balancer with both udp and tcp ports, and to reduce the attack surface by running K8GB on a non-privileged port.

Since version 1.26 Kubernetes supports load balancers with mixed ports.
The coredns helm chart also evolved over the years, and with this contribution we are able to not open privileged ports on coredns.

Generated configuration

The generated configuration by the base chart has only the following differences:

• Ports exposed by the coredns service (fixes CoreDNS AWS NLB health check not getting healthy #1741)

<   - {port: 53, targetPort: 5353, protocol: UDP, name: udp-5353}
---
>   - {"name":"udp-5353","port":53,"protocol":"UDP","targetPort":5353}
>   - {"name":"tcp-5353","port":53,"protocol":"TCP","targetPort":5353}

• Ports exposed by the coredns container (allows prometheus metrics scraping)

<         - {containerPort: 5353, protocol: UDP, name: udp-5353}
<         - {containerPort: 5353, protocol: TCP, name: tcp-5353}
---
>         - {"containerPort":5353,"name":"udp-5353","protocol":"UDP"}
>         - {"containerPort":5353,"name":"tcp-5353","protocol":"TCP"}
>         - {"containerPort":9153,"name":"tcp-9153","protocol":"TCP"}

• coredns container security context (to disable the NET_BIND_SERVICE capability)

>         securityContext:
>           capabilities:
>             add: []

User configuration

The helm configuration changes slightly, so users should be aware of the following:

• dnsZoneNegTTL is now configured directly on the coredns chart, on the k8s_crd block under coredns.servers.plugins
• coredns.extraServerBlocks and coredns.extraPlugins no longer exist and must be configured using coredns native configuration
• coredns.servers block must be configured by copying the base chart's values and adapting the zone key. This is unfortunately a usability drawback since the previous setup configured it on the k8gb chart by only using a couple of values. However, this is necessary to benefit from all the coredns's port logic that enables TCP and prometheus ports.

Conclusion

With this change we are now able to use the upstream chart instead of a fork. Unfortunately there is a overhead on the configuration necessary to the user, but we gain a lot. We can profit from all the features the coredns community offers, expose TCP port 53 publicly (fixes #1741) and expose the metrics port on the container which allows Prometheus metrics scraping.

---

Others

The upgrade from ubuntu 22.04 to 24.04 seems to have affected the python environment. We now need to create a virtual environment to enable installation of libraries via pip3.

By enabling k3d's loadbalancer we can rely on routing via Kubernetes services. This means we can use services of type LoadBalancer or ClusterIP. We no longer need to rely on the service type NodePort, or on making sure a pod is running on the node exposing the port. With this improvement we can now query the coredns servers using udp or tcp:

➜  ~ dig -p 5054 @localhost localtargets-roundrobin.cloud.example.com +short
172.19.0.11
172.19.0.12
➜  ~ dig -p 5054 @localhost localtargets-roundrobin.cloud.example.com +tcp +short
172.19.0.11
172.19.0.12

In a follow up PR I will try to extend this setup to expose istio and the nginx ingress controller.

[netlify]

✅ Deploy Preview for k8gb-preview ready!

Name	Link
🔨 Latest commit	b6a49af
🔍 Latest deploy log	https://app.netlify.com/sites/k8gb-preview/deploys/670f9139e4c5080008f4bf22
😎 Deploy Preview	https://deploy-preview-1743--k8gb-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[k0da]

@abaguas This is cool. It just an upstream chart never worked with ServiceType=LoadBalancer. It has a conflict due to this wasn't there. I never tested this FeatureGate

[abaguas]

@abaguas This is cool. It just an upstream chart never worked with ServiceType=LoadBalancer. It has a conflict due to this wasn't there. I never tested this FeatureGate

Thank you @k0da! I just tested it in my cluster and I can have a LoadBalancer service with both UDP and TCP