Recipe for searching similar videos

[plon-Susk7]

This PR is related to issue #410

1. Created folder called notebooks inside src.
2. Installed Jupyter lab inside virtual environment.
3. Made changes to Dockerfile and docker-compose.yml file to expose notebook's port.
4. Added torchvision in vid_vec_rep_clip.in in order to resolve error.
5. Command to run notebook outside container :

jupyter lab --ip 0.0.0.0 --port 8888 --no-browser --allow-root --NotebookApp.token=''

6. Created notebook for recipe.

[dennyabrain]

This is a good start! Especially creating a python notebook for these recipes is pretty neat.

We will have to make some fundamental changes to how you are doing the recipes. I'm going to write some incorrect code to demonstrate the intent.

Firstly we would like to provide functions in feluda to do some of the operations without exposing their details to the end user. For instance, we would like the user to define a config.yml file like

store :
  label : "Data Store"
  type : "es_vec"
  parameters:
    host_name : "es"
    image_index_name : "image"
    text_index_name : "text"
    video_index_name : "video"
    audio_index_name : "audio"

operators : 
  label : "Operators"
  parameters :
    - name : "Video Vector Representation"
      type : "vid_vec_rep_resnet"
      parameters: {
        index_name : "video"
      }

and then use Feluda like

feluda = Feluda("config.yml")
feluda.start() # starts store, server etc, if any in config.yml
for file in files:
    embedding = feluda.operators.vid_vec_rep_resnet.run(file)
    feluda.store(file, embedding)

result = feluda.search(files[2])
print(results)

Feluda's end user will not have to deal with details like get_vector_representation_of_video and es.connect() etc.
Ideally we'd like to hide these details as much as possible for a beginner user, but also provides ways for an expert user to access these lower level functions. So maybe something like feluda.stores.es.connect()

Its possible we will have to change some features in Feluda to make it work but thats the plan. We'll use this space to discuss this. Might take a while to come to a good solution, so bear with the uncertainty :)

[aatmanvaidya]

ohh right yes, my bad @dennyabrain, we should do it the way you suggested

@plon-Susk7 this is a great start!

• here is a config file for your reference - https://github.com/tattle-made/feluda/blob/main/src/worker/media/config.yml

• the second part on using Feluda class for everything you will find detailed code in this file - https://github.com/tattle-made/feluda/blob/main/src/worker/media/media_worker.py

  • here are the specific lines that might help you - init feluda and its components - download video file and extract embeddings - store in ES - you will have to write code to search for a file using Feluda

• just a check - if we are running the jupyter lab outside of docker, this means that the venv has to activated right?

[aatmanvaidya]

@dennyabrain most open source projects have a folder called notebooks or examples where they have example code usage of their framework/ library. Any specific preference on the naming? we can call it code recipes too.

Once we push Feluda as a pypi package, we will move this folder out of src

[dennyabrain]

@aatmanvaidya I don't have a strong preference. Anything thats familiar to the python ecosystem is preferable. If we are going to be using python notebooks, something like "notebooks" is definitely familiar than "code recipes".

[dennyabrain]

@plon-Susk7 also wanted to flag that if you see anything that doesn't work as expected or see code that seems odd, do flag it to us. A lot of stuff in feluda emerged either through design or us trying to make it work for some specific purposes. So there might be inconsistencies. A big goal of the 1.0 release is to fix these things. So feel free to flag them, discuss them and if possible fix too. All that effort will actually go in addressing #409

[plon-Susk7]

• just a check - if we are running the jupyter lab outside of docker, this means that the venv has to activated right?

Hey @aatmanvaidya , i did install Jupyter lab inside venv but I deactivated as soon as it was installed. Isn't it supposed to be that way according to the wiki?