ADE Select Model - Doesnt show the list of openai models #1897

raolak · 2024-10-16T10:40:02Z

Describe the bug
I have set environment variables while running docker compose in my local setup. But in the ADE UI I see only letta-free option under dropdown. My understanding if I set llm model for open ai then tha should get listed. Can you please help with steps to see all the model dropdowsn in the ADE UI while creating the agent?

Please describe your setup
FOllowed this document to setup letta using docker compose
https://docs.letta.com/install

Environment variables within docker container

env

LETTA_PG_DB=letta
LETTA_LLM_MODEL=gpt-4
HOSTNAME=letta
SERPAPI_API_KEY=
PYTHON_PIP_VERSION=24.0
HOME=/root
OLDPWD=/letta
LETTA_ENVIRONMENT=PRODUCTION
LETTA_LLM_ENDPOINT=https://api.openai.com/v1
LETTA_LLM_CONTEXT_WINDOW=None
OPENAI_API_KEY=
GPG_KEY=7169605F62C751356D054A26A821E680E5FA6305
LETTA_EMBEDDING_ENDPOINT_TYPE=openai
PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/dbf0c85f76fb6e1ab42aa672ffca6f0a675d9ee4/public/get-pip.py
TERM=xterm
PATH=/app/.venv/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
LETTA_EMBEDDING_MODEL=text-embedding-ada-002
LANG=C.UTF-8
LETTA_PG_PASSWORD=letta
LETTA_EMBEDDING_DIM=None
PYTHON_VERSION=3.12.2
LETTA_EMBEDDING_ENDPOINT=https://api.openai.com/v1
LETTA_PG_HOST=pgvector_db
LETTA_PG_USER=letta
VIRTUAL_ENV=/app/.venv
PWD=/root/.letta
PYTHON_GET_PIP_SHA256=dfe9fd5c28dc98b5ac17979a953ea550cec37ae1b47a5116007395bfacff2ab9
LETTA_LLM_ENDPOINT_TYPE=openai
LETTA_PG_PORT=5432

Startup logs
2024-10-16 15:04:53 uvicorn.access - INFO - 192.168.65.1:31061 - "GET /v1/models/embedding HTTP/1.1" 200
2024-10-16 15:04:53 uvicorn.access - INFO - 192.168.65.1:31066 - "GET /v1/blocks/?label=human HTTP/1.1" 200
2024-10-16 15:04:53 uvicorn.access - INFO - 192.168.65.1:31065 - "GET /v1/tools/ HTTP/1.1" 200
2024-10-16 15:04:53 uvicorn.access - INFO - 192.168.65.1:31064 - "GET /v1/blocks/?label=persona HTTP/1.1" 200
2024-10-16 15:04:53 uvicorn.access - INFO - 192.168.65.1:31063 - "GET /v1/agents/ HTTP/1.1" 200
2024-10-16 15:04:54 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)]

Complete logs
2024-10-16 15:02:45 Starting MEMGPT server...
2024-10-16 15:02:50 uvicorn.error - INFO - Started server process [8]
2024-10-16 15:02:50 uvicorn.error - INFO - Waiting for application startup.
2024-10-16 15:02:50 uvicorn.error - INFO - Application startup complete.
2024-10-16 15:02:50 uvicorn.error - INFO - Uvicorn running on http://0.0.0.0:8083 (Press CTRL+C to quit)
2024-10-16 15:03:17 uvicorn.access - INFO - 192.168.65.1:31051 - "GET / HTTP/1.1" 200
2024-10-16 15:03:17 uvicorn.access - INFO - 192.168.65.1:31051 - "GET /index-9a9c449b.js HTTP/1.1" 200
2024-10-16 15:03:17 uvicorn.access - INFO - 192.168.65.1:31050 - "GET /index-3ab03d5b.css HTTP/1.1" 200
2024-10-16 15:03:17 uvicorn.access - INFO - 192.168.65.1:31051 - "GET /favicon.ico HTTP/1.1" 404
2024-10-16 15:03:17 uvicorn.access - INFO - 192.168.65.1:31054 - "GET /v1/blocks/?label=human HTTP/1.1" 200
2024-10-16 15:03:17 uvicorn.access - INFO - 192.168.65.1:31053 - "GET /v1/blocks/?label=persona HTTP/1.1" 200
2024-10-16 15:03:17 uvicorn.access - INFO - 192.168.65.1:31052 - "GET /v1/tools/ HTTP/1.1" 200
2024-10-16 15:03:17 uvicorn.access - INFO - 192.168.65.1:31051 - "GET /v1/agents/ HTTP/1.1" 200
2024-10-16 15:03:18 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)]
2024-10-16 15:03:18 uvicorn.access - INFO - 192.168.65.1:31050 - "GET /v1/models/ HTTP/1.1" 200
2024-10-16 15:03:19 uvicorn.access - INFO - 192.168.65.1:31056 - "GET /v1/sources/ HTTP/1.1" 200
2024-10-16 15:03:19 uvicorn.access - INFO - 192.168.65.1:31055 - "GET /v1/jobs/active HTTP/1.1" 200
2024-10-16 15:03:20 uvicorn.access - INFO - 192.168.65.1:31055 - "GET /v1/agents/ HTTP/1.1" 200
2024-10-16 15:03:20 uvicorn.access - INFO - 192.168.65.1:31057 - "GET /v1/blocks/?label=human HTTP/1.1" 200
2024-10-16 15:03:20 uvicorn.access - INFO - 192.168.65.1:31059 - "GET /v1/blocks/?label=persona HTTP/1.1" 200
2024-10-16 15:03:20 uvicorn.access - INFO - 192.168.65.1:31058 - "GET /v1/tools/ HTTP/1.1" 200
2024-10-16 15:03:20 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)]
2024-10-16 15:03:20 uvicorn.access - INFO - 192.168.65.1:31056 - "GET /v1/models/ HTTP/1.1" 200
2024-10-16 15:03:21 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)]
2024-10-16 15:03:21 uvicorn.access - INFO - 192.168.65.1:31056 - "GET /v1/models/embedding HTTP/1.1" 200
2024-10-16 15:04:53 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)]
2024-10-16 15:04:53 uvicorn.access - INFO - 192.168.65.1:31061 - "GET /v1/models/embedding HTTP/1.1" 200
2024-10-16 15:04:53 uvicorn.access - INFO - 192.168.65.1:31066 - "GET /v1/blocks/?label=human HTTP/1.1" 200
2024-10-16 15:04:53 uvicorn.access - INFO - 192.168.65.1:31065 - "GET /v1/tools/ HTTP/1.1" 200
2024-10-16 15:04:53 uvicorn.access - INFO - 192.168.65.1:31064 - "GET /v1/blocks/?label=persona HTTP/1.1" 200
2024-10-16 15:04:53 uvicorn.access - INFO - 192.168.65.1:31063 - "GET /v1/agents/ HTTP/1.1" 200
2024-10-16 15:04:54 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)]
2024-10-16 15:04:54 uvicorn.access - INFO - 192.168.65.1:31062 - "GET /v1/models/ HTTP/1.1" 200
2024-10-16 15:27:51 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)]
2024-10-16 15:27:51 uvicorn.access - INFO - 192.168.65.1:31096 - "GET /v1/models/embedding HTTP/1.1" 200
2024-10-16 15:27:52 uvicorn.access - INFO - 192.168.65.1:31095 - "GET /v1/blocks/?label=persona HTTP/1.1" 200
2024-10-16 15:27:52 uvicorn.access - INFO - 192.168.65.1:31097 - "GET /v1/tools/ HTTP/1.1" 200
2024-10-16 15:27:52 uvicorn.access - INFO - 192.168.65.1:31098 - "GET /v1/blocks/?label=human HTTP/1.1" 200
2024-10-16 15:27:52 uvicorn.access - INFO - 192.168.65.1:31093 - "GET /v1/agents/ HTTP/1.1" 200
2024-10-16 15:27:53 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)]
2024-10-16 15:27:53 uvicorn.access - INFO - 192.168.65.1:31094 - "GET /v1/models/ HTTP/1.1" 200
2024-10-16 15:27:59 uvicorn.access - INFO - 192.168.65.1:31100 - "GET /v1/jobs/active HTTP/1.1" 200
2024-10-16 15:27:59 uvicorn.access - INFO - 192.168.65.1:31099 - "GET /v1/sources/ HTTP/1.1" 200
2024-10-16 15:28:00 uvicorn.access - INFO - 192.168.65.1:31101 - "GET /v1/blocks/?label=human HTTP/1.1" 200
2024-10-16 15:28:00 uvicorn.access - INFO - 192.168.65.1:31103 - "GET /v1/tools/ HTTP/1.1" 200
2024-10-16 15:28:00 uvicorn.access - INFO - 192.168.65.1:31099 - "GET /v1/blocks/?label=persona HTTP/1.1" 200
2024-10-16 15:28:00 uvicorn.access - INFO - 192.168.65.1:31100 - "GET /v1/agents/ HTTP/1.1" 200
2024-10-16 15:28:01 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)]
2024-10-16 15:28:01 uvicorn.access - INFO - 192.168.65.1:31100 - "GET /v1/models/embedding HTTP/1.1" 200
2024-10-16 15:28:01 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)]
2024-10-16 15:28:01 uvicorn.access - INFO - 192.168.65.1:31102 - "GET /v1/models/ HTTP/1.1" 200
2024-10-16 15:33:02 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)]
2024-10-16 15:33:02 uvicorn.access - INFO - 192.168.65.1:31121 - "GET /v1/models/embedding HTTP/1.1" 200
2024-10-16 15:33:02 uvicorn.access - INFO - 192.168.65.1:31119 - "GET /v1/blocks/?label=persona HTTP/1.1" 200
2024-10-16 15:33:02 uvicorn.access - INFO - 192.168.65.1:31120 - "GET /v1/blocks/?label=human HTTP/1.1" 200
2024-10-16 15:33:02 uvicorn.access - INFO - 192.168.65.1:31116 - "GET /v1/tools/ HTTP/1.1" 200
2024-10-16 15:33:02 uvicorn.access - INFO - 192.168.65.1:31117 - "GET /v1/agents/ HTTP/1.1" 200
2024-10-16 15:33:02 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)]
2024-10-16 15:33:02 uvicorn.access - INFO - 192.168.65.1:31118 - "GET /v1/models/ HTTP/1.1" 200
2024-10-16 15:33:04 uvicorn.access - INFO - 192.168.65.1:31123 - "GET /v1/jobs/active HTTP/1.1" 200
2024-10-16 15:33:04 uvicorn.access - INFO - 192.168.65.1:31122 - "GET /v1/sources/ HTTP/1.1" 200
2024-10-16 15:33:23 uvicorn.access - INFO - 192.168.65.1:31124 - "GET /v1/blocks/?label=human HTTP/1.1" 200
2024-10-16 15:33:25 uvicorn.access - INFO - 192.168.65.1:31125 - "GET /v1/blocks/?label=persona HTTP/1.1" 200
2024-10-16 15:33:46 uvicorn.access - INFO - 192.168.65.1:31126 - "GET /v1/tools/ HTTP/1.1" 200
2024-10-16 15:33:47 uvicorn.access - INFO - 192.168.65.1:31126 - "GET /v1/jobs/active HTTP/1.1" 200
2024-10-16 15:33:47 uvicorn.access - INFO - 192.168.65.1:31127 - "GET /v1/sources/ HTTP/1.1" 200
2024-10-16 15:33:49 uvicorn.access - INFO - 192.168.65.1:31128 - "GET /v1/tools/ HTTP/1.1" 200
2024-10-16 15:33:49 uvicorn.access - INFO - 192.168.65.1:31131 - "GET /v1/blocks/?label=human HTTP/1.1" 200
2024-10-16 15:33:49 uvicorn.access - INFO - 192.168.65.1:31132 - "GET /v1/agents/ HTTP/1.1" 200
2024-10-16 15:33:49 uvicorn.access - INFO - 192.168.65.1:31129 - "GET /v1/blocks/?label=persona HTTP/1.1" 200
2024-10-16 15:33:50 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)]
2024-10-16 15:33:50 uvicorn.access - INFO - 192.168.65.1:31130 - "GET /v1/models/ HTTP/1.1" 200
2024-10-16 15:33:51 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)]
2024-10-16 15:33:51 uvicorn.access - INFO - 192.168.65.1:31130 - "GET /v1/models/embedding HTTP/1.1" 200
2024-10-16 15:34:12 uvicorn.access - INFO - 192.168.65.1:31133 - "GET /v1/sources/ HTTP/1.1" 200
2024-10-16 15:34:12 uvicorn.access - INFO - 192.168.65.1:31134 - "GET /v1/jobs/active HTTP/1.1" 200
2024-10-16 15:34:13 uvicorn.access - INFO - 192.168.65.1:31133 - "GET /v1/blocks/?label=persona HTTP/1.1" 200
2024-10-16 15:34:13 uvicorn.access - INFO - 192.168.65.1:31137 - "GET /v1/blocks/?label=human HTTP/1.1" 200
2024-10-16 15:34:13 uvicorn.access - INFO - 192.168.65.1:31136 - "GET /v1/tools/ HTTP/1.1" 200
2024-10-16 15:34:13 uvicorn.access - INFO - 192.168.65.1:31134 - "GET /v1/agents/ HTTP/1.1" 200
2024-10-16 15:34:14 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)]
2024-10-16 15:34:14 uvicorn.access - INFO - 192.168.65.1:31135 - "GET /v1/models/ HTTP/1.1" 200
2024-10-16 15:34:15 uvicorn.access - INFO - 192.168.65.1:31135 - "GET /v1/jobs/active HTTP/1.1" 200
2024-10-16 15:34:15 uvicorn.access - INFO - 192.168.65.1:31138 - "GET /v1/sources/ HTTP/1.1" 200
2024-10-16 15:34:16 uvicorn.access - INFO - 192.168.65.1:31135 - "GET /v1/blocks/?label=persona HTTP/1.1" 200
2024-10-16 15:34:16 uvicorn.access - INFO - 192.168.65.1:31139 - "GET /v1/blocks/?label=human HTTP/1.1" 200
2024-10-16 15:34:16 uvicorn.access - INFO - 192.168.65.1:31140 - "GET /v1/tools/ HTTP/1.1" 200
2024-10-16 15:34:16 uvicorn.access - INFO - 192.168.65.1:31138 - "GET /v1/agents/ HTTP/1.1" 200
2024-10-16 15:34:17 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)]
2024-10-16 15:34:17 uvicorn.access - INFO - 192.168.65.1:31141 - "GET /v1/models/ HTTP/1.1" 200
2024-10-16 15:34:48 uvicorn.access - INFO - 192.168.65.1:31144 - "GET /v1/tools/ HTTP/1.1" 200
2024-10-16 15:34:48 uvicorn.access - INFO - 192.168.65.1:31143 - "GET /v1/blocks/?label=persona HTTP/1.1" 200
2024-10-16 15:34:48 uvicorn.access - INFO - 192.168.65.1:31145 - "GET /v1/blocks/?label=human HTTP/1.1" 200
2024-10-16 15:34:48 uvicorn.access - INFO - 192.168.65.1:31142 - "GET /v1/agents/ HTTP/1.1" 200
2024-10-16 15:34:49 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)]
2024-10-16 15:34:49 uvicorn.access - INFO - 192.168.65.1:31146 - "GET /v1/models/ HTTP/1.1" 200
2024-10-16 15:41:24 uvicorn.access - INFO - 192.168.65.1:31151 - "GET /v1/blocks/?label=human HTTP/1.1" 200
2024-10-16 15:41:24 uvicorn.access - INFO - 192.168.65.1:31148 - "GET /v1/agents/ HTTP/1.1" 200
2024-10-16 15:41:24 uvicorn.access - INFO - 192.168.65.1:31147 - "GET /v1/tools/ HTTP/1.1" 200
2024-10-16 15:41:24 uvicorn.access - INFO - 192.168.65.1:31150 - "GET /v1/blocks/?label=persona HTTP/1.1" 200
2024-10-16 15:41:25 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)]
2024-10-16 15:41:25 uvicorn.access - INFO - 192.168.65.1:31149 - "GET /v1/models/ HTTP/1.1" 200
2024-10-16 15:41:26 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)]
2024-10-16 15:41:26 uvicorn.access - INFO - 192.168.65.1:31149 - "GET /v1/models/embedding HTTP/1.1" 200
2024-10-16 15:41:37 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)]
2024-10-16 15:41:37 uvicorn.access - INFO - 192.168.65.1:31152 - "GET /v1/models/embedding HTTP/1.1" 200
2024-10-16 15:42:29 Added record with id message-5f5e919f-9b8c-4947-9551-a690095f4bb1
2024-10-16 15:42:29 Added record with id message-34ec5961-5eb7-419d-9e84-f0b278e637e1
2024-10-16 15:42:29 Added record with id message-1a6d60b3-d6da-4029-aa8e-38a3860162d2
2024-10-16 15:42:29 Added record with id message-234f4672-f380-4d20-b5bb-9938493afed2
2024-10-16 15:42:29 Added record with id message-3cc56703-6901-4af7-8e27-7949b0ef5814
2024-10-16 15:42:29 uvicorn.access - INFO - 192.168.65.1:31153 - "POST /v1/agents/ HTTP/1.1" 200
2024-10-16 15:42:29 uvicorn.access - INFO - 192.168.65.1:31153 - "GET /v1/agents/ HTTP/1.1" 200
2024-10-16 15:43:33 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)]
2024-10-16 15:43:33 uvicorn.access - INFO - 192.168.65.1:31154 - "GET /v1/models/embedding HTTP/1.1" 200
2024-10-16 15:43:33 uvicorn.access - INFO - 192.168.65.1:31156 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/sources HTTP/1.1" 200
2024-10-16 15:43:33 uvicorn.access - INFO - 192.168.65.1:31159 - "GET /v1/tools/ HTTP/1.1" 200
2024-10-16 15:43:33 uvicorn.access - INFO - 192.168.65.1:31158 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/messages?limit=1000&msg_object=true HTTP/1.1" 200
2024-10-16 15:43:33 Letta.letta.server.server - ERROR - Agent (user=user-00000000, agent=agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96) is already loaded
2024-10-16 15:43:33 NoneType: None
2024-10-16 15:43:33 uvicorn.access - INFO - 192.168.65.1:31157 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96 HTTP/1.1" 200
2024-10-16 15:43:33 Letta.letta.server.server - ERROR - Agent (user=user-00000000, agent=agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96) is already loaded
2024-10-16 15:43:33 NoneType: None
2024-10-16 15:43:33 uvicorn.access - INFO - 192.168.65.1:31154 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/archival HTTP/1.1" 200
2024-10-16 15:43:34 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)]
2024-10-16 15:43:34 uvicorn.access - INFO - 192.168.65.1:31155 - "GET /v1/models/ HTTP/1.1" 200
2024-10-16 15:43:40 Warning: token streaming is only supported for OpenAI models. Setting to False.
2024-10-16 15:43:40 uvicorn.access - INFO - 192.168.65.1:31160 - "POST /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/messages HTTP/1.1" 200
2024-10-16 15:43:40 Added record with id message-2041d1ce-6231-4504-8431-06eb170871bc
2024-10-16 15:43:45 Added record with id message-297022c9-d6be-4b4e-ab18-1c17ab46b314
2024-10-16 15:43:45 Added record with id message-60080407-5148-4578-b18d-5afaa85b45fc
2024-10-16 15:43:45 Added record with id message-790a4dfe-eb4d-4f0b-8d11-141d13b3b97b
2024-10-16 15:44:04 uvicorn.access - INFO - 192.168.65.1:31161 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/sources HTTP/1.1" 200
2024-10-16 15:44:04 uvicorn.access - INFO - 192.168.65.1:31162 - "GET /v1/sources/ HTTP/1.1" 200
2024-10-16 16:00:44 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)]
2024-10-16 16:00:44 uvicorn.access - INFO - 192.168.65.1:31186 - "GET /v1/models/embedding HTTP/1.1" 200
2024-10-16 16:00:44 uvicorn.access - INFO - 192.168.65.1:31186 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/sources HTTP/1.1" 200
2024-10-16 16:00:44 uvicorn.access - INFO - 192.168.65.1:31185 - "GET /v1/tools/ HTTP/1.1" 200
2024-10-16 16:00:44 uvicorn.access - INFO - 192.168.65.1:31184 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96 HTTP/1.1" 200
2024-10-16 16:00:44 uvicorn.access - INFO - 192.168.65.1:31183 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/archival HTTP/1.1" 200
2024-10-16 16:00:44 uvicorn.access - INFO - 192.168.65.1:31187 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/messages?limit=1000&msg_object=true HTTP/1.1" 200
2024-10-16 16:00:45 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)]
2024-10-16 16:00:45 uvicorn.access - INFO - 192.168.65.1:31182 - "GET /v1/models/ HTTP/1.1" 200
2024-10-16 16:01:40 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)]
2024-10-16 16:01:40 uvicorn.access - INFO - 192.168.65.1:31200 - "GET /v1/models/embedding HTTP/1.1" 200
2024-10-16 16:01:40 uvicorn.access - INFO - 192.168.65.1:31200 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/sources HTTP/1.1" 200
2024-10-16 16:01:40 uvicorn.access - INFO - 192.168.65.1:31201 - "GET /v1/tools/ HTTP/1.1" 200
2024-10-16 16:01:40 uvicorn.access - INFO - 192.168.65.1:31205 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/archival HTTP/1.1" 200
2024-10-16 16:01:40 uvicorn.access - INFO - 192.168.65.1:31204 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96 HTTP/1.1" 200
2024-10-16 16:01:40 uvicorn.access - INFO - 192.168.65.1:31203 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/messages?limit=1000&msg_object=true HTTP/1.1" 200
2024-10-16 16:01:41 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)]
2024-10-16 16:01:41 uvicorn.access - INFO - 192.168.65.1:31202 - "GET /v1/models/ HTTP/1.1" 200
2024-10-16 16:01:45 uvicorn.access - INFO - 192.168.65.1:31210 - "GET /v1/blocks/?label=persona HTTP/1.1" 200
2024-10-16 16:01:45 uvicorn.access - INFO - 192.168.65.1:31206 - "GET /v1/tools/ HTTP/1.1" 200
2024-10-16 16:01:45 uvicorn.access - INFO - 192.168.65.1:31207 - "GET /v1/blocks/?label=human HTTP/1.1" 200
2024-10-16 16:01:45 uvicorn.access - INFO - 192.168.65.1:31208 - "GET /v1/agents/ HTTP/1.1" 200
2024-10-16 16:01:46 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)]
2024-10-16 16:01:46 uvicorn.access - INFO - 192.168.65.1:31208 - "GET /v1/models/embedding HTTP/1.1" 200
2024-10-16 16:01:46 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)]
2024-10-16 16:01:46 uvicorn.access - INFO - 192.168.65.1:31209 - "GET /v1/models/ HTTP/1.1" 200

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

Letta Config
Please attach your ~/.letta/config file or copy past it below.

Appreciate if any one can help here...this is blocking demo :(

The text was updated successfully, but these errors were encountered:

sarahwooders · 2024-10-16T19:15:02Z

Can you make sure you have the latest docker image? You can run docker pull lettaai/letta:latest. I just tried with OpenAI and wasn't able to reproduce the issue, so this might be a problem with an older version?

Also for your enviornment variables, you should also need to set OPENAI_API_KEY. The following variables are deprecated so won't be read/used:

LETTA_LLM_MODEL=gpt-4
LETTA_LLM_ENDPOINT=https://api.openai.com/v1
LETTA_LLM_CONTEXT_WINDOW=None
LETTA_EMBEDDING_ENDPOINT_TYPE=openai

LETTA_EMBEDDING_MODEL=text-embedding-ada-002
LETTA_EMBEDDING_DIM=None
LETTA_EMBEDDING_ENDPOINT=https://api.openai.com/v1
LETTA_LLM_ENDPOINT_TYPE=openai

Also if you have an older version of the DB initialized, you may need to run rm -r ./.persist/pgdata.

raolak · 2024-10-17T03:36:48Z

Hi Saraha, Thank you for the quick response—I really appreciate it! My goal is to deploy the Letta server on Kubernetes. Below are some challenges I encountered during the process. I managed to get the server running on Kubernetes earlier (before the rebranding), but I’m now reinstalling it with the latest version. Here's what I’ve done so far: 1. I initially tried using the lettaai/letta:latest image, but it seems incompatible with my Mac microk8s architecture. Could you confirm if this image supports all architectures? If needed, I’m happy to help create a compatible version. 2. I built the image from the repo’s Dockerfile and pushed it to my local registry. 3. My Helm chart currently copies config and secret files to /root/.letta. I’m unsure if this step is still necessary with the latest version. 4. I pass all necessary environment variables, including DB-specific ones, through the deployment YAML. I suspect I missed setting the database name, which is causing Letta to default to a local SQLite DB—I'm investigating this. Here’s the environment configuration I’m using inside the container: LETTA_LLM_MODEL=gpt-4o LETTA_ENVIRONMENT=PRODUCTION LETTA_LLM_CONTEXT_WINDOW=8192 OPENAI_API_KEY=sk-xxx LETTA_LLM_ENDPOINT=https://api.openai.com/v1 LETTA_EMBEDDING_CHUNK_SIZE=300 LETTA_EMBEDDING_ENDPOINT_TYPE=openai LETTA_DEBUG=true LETTA_EMBEDDING_MODEL=text-embedding-ada-002 LETTA_PG_HOST=postgres-db (k8s service name) LETTA_PG_PASSWORD=xxx LETTA_PG_USER=letta LETTA_PG_PORT=5432 PYTHON_VERSION=3.12.2 LETTA_EMBEDDING_DIM=1536 LETTA_EMBEDDING_ENDPOINT=https://api.openai.com/v1 LETTA_SERVER_PASS=xxx LETTA_LLM_ENDPOINT_TYPE=openai Clarifications 1. Do I need to deploy a separate server for each model, with corresponding environment variables for each? 2. Is there a way to configure models without relying on environment variables? 3. I’m exploring the use of the data source feature to generate boilerplate schemas. Is this feasible? We can dive deeper into this after I sort out other issues. I’d love to have a quick call to discuss my platform use cases. Letta is a critical enabler for my platform, and I appreciate all the great work your team is doing🙏. I’m also happy to contribute to any efforts that help with improvements or fixes. Thanks again! Best, Inr

…

On Thu, Oct 17, 2024 at 12:45 AM Sarah Wooders ***@***.***> wrote: Can you make sure you have the latest docker image? You can run docker pull lettaai/letta:latest. I just tried with OpenAI and wasn't able to reproduce the issue, so this might be a problem with an older version? Also for your enviornment variables, you should also need to set OPENAI_API_KEY. The following variables are deprecated so won't be read/used: LETTA_LLM_MODEL=gpt-4 LETTA_LLM_ENDPOINT=https://api.openai.com/v1 LETTA_LLM_CONTEXT_WINDOW=None LETTA_EMBEDDING_ENDPOINT_TYPE=openai <https://api.openai.com/v1LETTA_LLM_CONTEXT_WINDOW=NoneLETTA_EMBEDDING_ENDPOINT_TYPE=openai> LETTA_EMBEDDING_MODEL=text-embedding-ada-002 LETTA_EMBEDDING_DIM=None LETTA_EMBEDDING_ENDPOINT=https://api.openai.com/v1 LETTA_LLM_ENDPOINT_TYPE=openai <https://api.openai.com/v1LETTA_LLM_ENDPOINT_TYPE=openai> Also if you have an older version of the DB initialized, you may need to run rm -r ./.persist/pgdata. — Reply to this email directly, view it on GitHub <#1897 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGPZOKK3AWXSEVVGWGZJKLZ323MZAVCNFSM6AAAAABQBE27WGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJXG42DANZYHA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

raolak · 2024-10-17T03:38:42Z

Can you make sure you have the latest docker image? You can run docker pull lettaai/letta:latest. I just tried with OpenAI and wasn't able to reproduce the issue, so this might be a problem with an older version?

Also for your enviornment variables, you should also need to set OPENAI_API_KEY. The following variables are deprecated so won't be read/used:
LETTA_LLM_MODEL=gpt-4
LETTA_LLM_ENDPOINT=https://api.openai.com/v1
LETTA_LLM_CONTEXT_WINDOW=None
LETTA_EMBEDDING_ENDPOINT_TYPE=openai

LETTA_EMBEDDING_MODEL=text-embedding-ada-002
LETTA_EMBEDDING_DIM=None
LETTA_EMBEDDING_ENDPOINT=https://api.openai.com/v1
LETTA_LLM_ENDPOINT_TYPE=openai
Also if you have an older version of the DB initialized, you may need to run rm -r ./.persist/pgdata.

I will re install again today using these steps and will keep you posted. Thank you !

raolak · 2024-10-17T07:44:50Z

I re installed with setting right environment variables...I was able to deploy letta in k8s... As mentioned, I have built the docker image and not used lettaai/letta:latest from docker hub as it had architecture compatibility issue

But I dont see gpt-4 model listing in the ADE create agent UI.

Here is :LETTA and OPEN AI specific the env from k8s pod

LETTA_EMBEDDING_MODEL=text-embedding-ada-002
LETTA_EMBEDDING_ENDPOINT_TYPE=openai
LETTA_EMBEDDING_CHUNK_SIZE=300
LETTA_EMBEDDING_ENDPOINT=https://api.openai.com/v1
LETTA_EMBEDDING_DIM=None

LETTA_LLM_MODEL=gpt-4
LETTA_LLM_CONTEXT_WINDOW=None
LETTA_LLM_ENDPOINT_TYPE=openai
LETTA_LLM_ENDPOINT=https://api.openai.com/v1

LETTA_ENVIRONMENT=PRODUCTION
LETTA_DEBUG=true

OPENAI_API_KEY=

LETTA_PG_DB=jnaa-ai
LETTA_PG_USER=jnaa
LETTA_PG_HOST=jnaa-postgres-jnaa-postgres-db
LETTA_PG_PORT=5432
LETTA_PG_PASSWORD=WsxQaz21!

POSTGRES_URI=postgresql+pg8000://jnaa:WsxQaz21!@jnaa-postgres-jnaa-postgres-db:5432/jnaa-ai

Also /root/.letta. configs and credentials

/root/.letta/config
[defaults]
preset = memgpt_chat
persona = sam_pov
human = basic

[archival_storage]
type = postgres
path = /root/.letta/chroma
uri = postgresql+pg8000://jnaa:XXXX!@jnaa-postgres-jnaa-postgres-db:5432/jnaa-ai

[recall_storage]
type = postgres
path = /root/.letta
uri = postgresql+pg8000://jnaa:XXX!@jnaa-postgres-jnaa-postgres-db:5432/jnaa-ai

[metadata_storage]
type = sqlite
path = /root/.letta

[version]
letta_version = 0.4.1

[client]
anon_clientid = user-00000000

/root/.letta/credentials

[openai]
auth_type = bearer_token
key = sk-....

[azure]
auth_type = api_key

Clarification

why [metadata_storage] still been using sqlite? How do i use postgres instead of sqlite?

sarahwooders self-assigned this Oct 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ADE Select Model - Doesnt show the list of openai models #1897

ADE Select Model - Doesnt show the list of openai models #1897

raolak commented Oct 16, 2024

sarahwooders commented Oct 16, 2024

raolak commented Oct 17, 2024 via email

raolak commented Oct 17, 2024

raolak commented Oct 17, 2024 •

edited by sarahwooders

Loading

ADE Select Model - Doesnt show the list of openai models #1897

ADE Select Model - Doesnt show the list of openai models #1897

Comments

raolak commented Oct 16, 2024

env

sarahwooders commented Oct 16, 2024

raolak commented Oct 17, 2024 via email

raolak commented Oct 17, 2024

raolak commented Oct 17, 2024 • edited by sarahwooders Loading

Here is :LETTA and OPEN AI specific the env from k8s pod

Also /root/.letta. configs and credentials

raolak commented Oct 17, 2024 •

edited by sarahwooders

Loading