Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark tool fails #1748

Open
2 tasks done
kaosbeat opened this issue Sep 12, 2024 · 0 comments
Open
2 tasks done

Benchmark tool fails #1748

kaosbeat opened this issue Sep 12, 2024 · 0 comments
Assignees

Comments

@kaosbeat
Copy link

kaosbeat commented Sep 12, 2024

Describe the bug
Trying to get MemGPT up and running it its most basic form, I added an ollama endpoint (tested) and local storage options for all applicable. Running memgpt benchmark I'm faced with :


[nltk_data] Downloading package punkt_tab to
[nltk_data]     /home/kaos/.pyenv/versions/3.10.6/lib/python3.10/site-
[nltk_data]     packages/llama_index/core/_static/nltk_cache...
[nltk_data]   Package punkt_tab is already up-to-date!

Depending on your hardware, this may take up to 30 minutes. This will also create 15 new agents.

version = 0.3.25
Traceback (most recent call last):
  File "/home/kaos/.pyenv/versions/3.10.6/bin/memgpt", line 8, in <module>
    sys.exit(app())
  File "/home/kaos/.pyenv/versions/3.10.6/lib/python3.10/site-packages/typer/main.py", line 328, in __call__
    raise e
  File "/home/kaos/.pyenv/versions/3.10.6/lib/python3.10/site-packages/typer/main.py", line 311, in __call__
    return get_command(self)(*args, **kwargs)
  File "/home/kaos/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/kaos/.pyenv/versions/3.10.6/lib/python3.10/site-packages/typer/core.py", line 783, in main
    return _main(
  File "/home/kaos/.pyenv/versions/3.10.6/lib/python3.10/site-packages/typer/core.py", line 225, in _main
    rv = self.invoke(ctx)
  File "/home/kaos/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/kaos/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/kaos/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/kaos/.pyenv/versions/3.10.6/lib/python3.10/site-packages/typer/main.py", line 683, in wrapper
    return callback(**use_params)  # type: ignore
  File "/home/kaos/.pyenv/versions/3.10.6/lib/python3.10/site-packages/memgpt/benchmark/benchmark.py", line 60, in bench
    agent = client.create_agent(
TypeError: LocalClient.create_agent() got an unexpected keyword argument 'persona'

Please describe your setup

  • How did you install memgpt?
    • pip install pymemgpt? pip install pymemgpt-nightly? git clone?
  • Describe your setup
    • What's your OS (Windows/MacOS/Linux)?
    • How are you running memgpt? (cmd.exe/Powershell/Anaconda Shell/Terminal)

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

MemGPT Config
Please attach your ~/.memgpt/config file or copy past it below.
openapi_memgpt.json


If you're not using OpenAI, please provide additional information on your local LLM setup:

Local LLM details
ollama using qwen2:0.5b
localhost API calls using curl work fine

If you are trying to run MemGPT with local LLMs, please provide the following information:

  • [qwen2:0.5b ] The exact model you're trying to use (e.g. dolphin-2.1-mistral-7b.Q6_K.gguf)
  • [ ollama] The local LLM backend you are using (web UI? LM Studio?)
  • [ debian12, 4090, local computer] Your hardware for the local LLM backend (local computer? operating system? remote RunPod?)
@sarahwooders sarahwooders self-assigned this Sep 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: To triage
Development

No branches or pull requests

2 participants