Create Recipes for common use case

[dennyabrain]

Recipes are supposed to be end to end examples of using Feluda for a particular use case. While feluda itself should be easy to use and configured by an experienced python/ML engineer, these recipes would provide easy to copy-paste examples of using feluda for specific use cases. As part of this issue, we should

1. Identify commonly useful recipes
2. Write example recipes
3. discover and fix bugs that we discover along the way
4. publish them in the wiki/website

[dennyabrain]

Examples of commonly useful recipes would be

1. Index a collection of images and search similar images in them
2. Index a collection of videos and search for similar videos through them
3. Index a collection of audios and search for similar audios through them
4. Process a collection of videos and cluster them into predefined categories
5. Process a collection of newspaper clippings and search through the text in them

[plon-Susk7]

Hi @dennyabrain , should we start with the examples provided by you first?

[dennyabrain]

Yes @plon-Susk7 let's do one of them. Video is interesting and sufficiently complex. Should we do the recipes related to videos?
So 1. Demonstrating the use of feluda to index and search through videos and 2. Demonstrating the use of feluda to cluster a collection of videos into groups.

@aatmanvaidya might be able to direct you better to which operaters to look at and any relevant documentation.

[plon-Susk7]

Yeah let's go with video first. I'll fetch additional details from @aatmanvaidya .

[plon-Susk7]

Steps to run notebook:

1. Install jupyter lab inside virtual environment first (venv).

pip install jupyter lab

2. After installation of jupyter lab, deactivate venv and run the following command.

jupyter lab --ip 0.0.0.0 --port 8888 --no-browser --allow-root --NotebookApp.token=''

You can run notebook by navigating to http://localhost:8888 on your browser.

[dennyabrain]

@aatmanvaidya @plon-Susk7 something to think about, should we considering creating collab notebooks as well? I have two reasons to support this :

1. It will provide journalists and non tech folks a cloud environment to use feluda in without worrying about installing python and feluda on their machine
2. since collab integrates well with google drive, they could mount data from their own drives.

2 is useful because a lot of people are familiar with using google drive and create their little personal "datasets" on it all the time. So being able to process their own data using our notebooks would enhance feluda's usability for them.

[aatmanvaidya]

@dennyabrain even I was thinking about this

but currently, there could be some limitations

• we will only be able to use operators in google colab.
  • this is not bad, because clustering, t-SNE, extraction of text from image and lot of useful things can still be done. Only store and search won't work
• elastic search won't work there -- we could think of using other vector databases from langchain etc, but that's a different discussion.
• since feluda is not a python package yet, we will have to clone the entire repo on the google coalb (this is not a big deal), but this means that whatever operator a journalist/not tech person would want to use, they would have to manually install there, and other dependencies that could come with it like ffmpeg, tessarct-ocr etc

I think we should definitely have examples on google colab, as its just becomes one-click for someone to replicate and use feluda -- they don't have to worry about setting up docker etc.

But I feel, through the work Priyash has done so far on writing example notebooks, we should first finalise the public API and then move towards examples on colab.
What do you think Denny?

[dennyabrain]

@aatmanvaidya point taken about the need to publish the library first and to finalize the API. Was getting ahead of myself. Lets do collab later then.