serving-dinov2

Contents

• Serving dinov2 onnx model with triton. (Artifact throughput: 48.927 infer/sec)
• Gradio Demo.
• Docker Compose.
• K8s Setting(Triton, Traefik, Promtail, Loki, Prometheus, Grafana).
• Serving dinov2 TensorRT Model. (Artifact throughput: 222.66 infer/sec)
• Serving dinov2 onnx model with Fastertransformer (fastertransformer_backend don't support vit yet.)

Docker Compose

Check docker-compose

Kubernetes

Check Kubernetes