What API / spec is this serving / using?

[cdrage]

When using this, I'm confused what API / spec it uses?

Is this a drop-in replacement for ollama? It is openapi spec compatible? is it ollamaspec? unsure what spec this is offering so I can use it as a drop-in replacement container for ollama.

[rhatdan]

The goal is to be a drop-in replacement for ollama using lama.cpp in the same way. We eventually want to expand to other runtimes like vllm.

We want to support pulling and pushing models from multiple Image Registries, which I am thinking of calling transports.

huggingface, ollama and OCI

These currently is not a lot of code here. Just mainly wrappers around other tools.

[ericcurtin]

Ollama models can be pulled, for inferencing, it's all llama.cpp, although it's likely we will add vllm as a backend also