Feature request: JSON mode output

[eliranwong]

Would you consider to support JSON mode output, just like Llama.cpp, Ollama, and OpenAI do?

e.g. https://llama-cpp-python.readthedocs.io/en/latest/#json-and-json-schema-mode

It is very limited in usage for building ai applications without JSON output implementation. Please kindly consider, thanks.

[kevinintel]

Thanks for the reminding, we will investigate into it