Does kbatch *require* kbatch-proxy?

[yuvipanda]

It would be awesome if there was a way to make kbatch work without kbatch-proxy. This throws the security model for a toss a little, but would allow powerful use cases where there's just one user per cluster. You can spin up a k8s cluster while working locally, run some jobs on it, and then tear down the cluster - making them truly ephemeral. This becomes a lot more difficult when there are serverside components required for it to work.

[TomAugspurger]

The job created in

kbatch/kbatch/kbatch/_backend.py

Line 42 in 083ebb4

def make_job(

should be mostly valid. Right now, the kbatch-proxy component does do a bit https://github.com/kbatch-dev/kbatch/blob/main/kbatch-proxy/kbatch_proxy/patch.py to update the job which includes

1. setting the namespace of each object to the user's namespace
2. adding the configmap, volumes, and containers to handle local code

I suspect that both of these could be moved client side. We would just need the server-side to verify that the namespace is actually correct?

My motivation for putting the volume / configmap stuff on the server side was because IIUC the ConfigMap has to exist (with a name) before it can be used as a volume. So we have to create that first, then update the volumes to refer to it by its name.

[TomAugspurger]

Just wanted to note that I 100% agree with the goal:

It would be awesome if there was a way to make kbatch work without kbatch-proxy.

So if we can sort out these issues and have the proxy just validate, rather than patch, stuff that'd be great.

[minrk]

I think if we connect this to #6, we can simplify it. If the client picks a reasonably unique name, it can either:

1. create job, then
2. create configmap/secret with ownership references

or

1. create all 3
2. patch configmap/secret with ownership references (slightly higher risk of orphan)

We should be able to handle name conflicts fine, since they will fail to create with kubernetes 409 conflict.

It is okay to create a Job which references a configmap that doesn't exist. It just won't start until the configmap is found and ready to mount.

With this approach, I think the client can create the whole thing, and kube-proxy should only need to:

1. enforce the namespace
2. execute the api requests to create resources
3. applying ownership references, which is the only step that has to be done after the Job is created

Effectively, that moves ~all of kbatch_proxy.patch into kbatch.

We might even be able to do something clever and accept an arbitrary list of resources, a la kubectl apply. The only trick I think is mapping a Spec to a create_namespaced_whatever method, but maybe that exists? That can wait until someone wants it.

[yuvipanda]

Do you think https://kubernetes.io/docs/reference/using-api/server-side-apply/ may help here, if we are to try doing things similar to kubectl apply?