Mounting Custom CAs on Controller

[vijay-wandb]

Description

Today: It is possible to mount custom CAs on the W&B application, but not the controller itself. Customers like Continental have SSL interceptors between Kubernetes and the internet and need custom CAs on everything that talks to the internet. Contacting deploy.wandb.ai therefore fails.

Expectation: Custom CAs can be mounted to the controller pod in the same way that they can be mounted to the application.

Marc-Steffen Kaesz
August 14, 2024 at 5:00 AM
Edited

As a workaround:

We created a config map with the CA via:

kubectl create configmap custom-ca-pemstore --from-file=ContinentalCorporateITSecurity-ContinentalAG.crt

We edited the deployment manually to unblock them:

spec:
  volumes:
  - name: custom-ca-pemstore
    configMap:
      name: custom-ca-pemstore
  containers:
  - volumeMounts:
    - name: custom-ca-pemstore
      mountPath: /etc/ssl/certs/ContinentalCorporateITSecurity-ContinentalAG.crt
      subPath: ContinentalCorporateITSecurity-ContinentalAG.crt
      readOnly: true
    env:
    - name: SSL_KEY_FILE
      value: /etc/ssl/certs/ContinentalCorporateITSecurity-ContinentalAG.crt

Controller is based on gcr.io/distroless/static-debian11. We cannot mount the bundle in the same way as on wandb-app as the update-ca-certificates command is not available.

[abhinavg6]

We need to close this. @flamarion has a PR - wandb/helm-charts#206