Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Newly created Collection available for all the users when authentication is enabled #1867

Open
llmwesee opened this issue Oct 3, 2024 · 3 comments

Comments

@llmwesee
Copy link

llmwesee commented Oct 3, 2024

When authentication is enabled, I want to create a shared collection, such as UserData, that remains accessible to all users at all times, regardless of who created it.

I encountered an issue when attempting to create a new collection (UserData2). Here’s the process I followed:

  1. I first created the collection using the UI.
  2. I then ran the following command:

python generate.py --base_model=meta-llama/Llama-2-13b-chat-hf --score_model=None --langchain_modes="['UserData',UserData2,LLM,MyData]" --user_path=user_path --use_auth_token=True --max_seq_len=4096 --max_max_new_tokens=2048 --max_new_tokens=2048 --min_new_tokens=128 --prompt_type=llama2 --enable_stt=False --enable_tts=False --auth_filename=$auth_filename --auth_access=open --guest_name=avser --auth="[(admin, admin)]"

The issue is that the newly created collection (UserData2) is only accessible to the user who created it, similar to how MyData functions. However, I would like this collection to behave like UserData, where it is available to all users globally, even when authentication is enabled.

However When authentication is disabled during collection creation, the collection is accessible to all users as expected.

so Could you please provide guidance on how to create a shared collection like UserData that remains accessible to all users, even when authentication is enabled?

@pseudotensor
Copy link
Collaborator

How did you create it in the UI? The box takes a few args, including the collection type (assumed to be personal if no passed).

@llmwesee
Copy link
Author

llmwesee commented Oct 4, 2024

Document Selection >> Add Collection then type UserData2, shared, userpath then put UserData2 in the
--langchain_modes="['UserData',UserData2,LLM,MyData]"

@llmwesee
Copy link
Author

llmwesee commented Oct 10, 2024

However i also created the collection with src/make_db.py: by adding all the files in the folder user_path3 then
python src/make_db.py --user_path=user_path3 --collection_name=UserData3 --langchain_type=shared

python generate.py --base_model=meta-llama/Llama-2-13b-chat-hf --score_model=None --langchain_mode='UserData' --user_path=user_path --use_auth_token=True --max_seq_len=4096 --max_max_new_tokens=2048 --batch_size=16 --prompt_type=llama2 --langchain_modes=['UserData','UserData3','MyData','LLM'] --auth_filename=$auth_filename --auth_access=open --guest_name=avser --auth="[(admin, admin)]"

then still didn't showing UserData3 in the collections for all the users although the embedding are stored in db_dir_UserData3 folder

And when adding --langchain_modes=['UserData','UserData3'] --langchain_mode_paths={'UserData':'user_path','UserData3':'user_path3'} --langchain_mode_types={'UserData':'shared','UserData3':'shared'}

to the command like:

python generate.py --base_model=meta-llama/Llama-2-13b-chat-hf --score_model=None --langchain_mode='UserData' --user_path=user_path --use_auth_token=True --max_seq_len=4096 --max_max_new_tokens=2048 --batch_size=16 --prompt_type=llama2 --langchain_modes=['UserData','UserData3'] --langchain_mode_paths={'UserData':'user_path','UserData3':'user_path3'} --langchain_mode_types={'UserData':'shared','UserData3':'shared'} --auth_filename=$auth_filename --auth_access=open --guest_name=avser --auth="[(admin, admin)]"

then it showing the following error:

File "/home/xxxx/src/gen.py", line 1383, in main langchain_mode_paths = str_to_dict(langchain_mode_paths) File "/home/xxxx/src/utils.py", line 1863, in str_to_dict raise ValueError("Invalid str_to_dict for %s" % x) ValueError: Invalid str_to_dict for UserData3:user_path3

Note: I created the authentication server through the LDAP
@pseudotensor please help me regarding this!

@h2oai h2oai deleted a comment Oct 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants