-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tasks stuck in processing
state, lots of tasks
#4743
Comments
Hey @0xMostafa, is it possible that meilisearch is crashing and restarting automatically? |
Hello here As @irevoire asked, do you see Meilisearch crashing and restarting? Do you have the same issue with v1.9.0? Also, do you have the same issue without Do you have a reproducible example, or an access to machine to share (in private) to help us debug this? |
when generating documents in large quantities,meilisearch exit and restart |
What's the resources of your machines? (RAM CPU). Let me know if it helps |
4 cores 8g、Will restarting the task not continue if it exits abnormally during task execution? Will a restart that has already been queued become in progress and still not execute? |
I'm not sure I get your question. Did you try to increase the RAM? Did you try with |
What is stuck? What are their status? Use this route: https://www.meilisearch.com/docs/reference/api/tasks#get-tasks If Meiliseach crashes and restart, it will try to re-process the current. But if the tasks is too big for the machine resources, it will crash in loop again and again. Did you try to increase the RAM? Did you try with --max-indexing-memory 0? How many documents do you have? How many do you try to index in the stuck task? |
Can you try to reduce the number of tasks process at the same time: with |
Thank you, I'm looking forward to the official version of the parameter to be added to https://www.meilisearch.com/docs/learn/self_hosted/configure_meilisearch_at_launch This is a very useful tool compared to controlling memory --max-indexing-memory. |
Yes sorry, I should have suggested it first! I did not realized the batch queue was too big! I opened an issue here: meilisearch/documentation#2958 |
I close this issue since it seems to be fixed! Let me know if it's not |
Describe the bug
I've run an indexing job of ~115k docs on meilisearch , now it has ~76k successfully indexed and
~35k tasks of type
documentAdditionOrUpdate
(inprocessing
state)it has been a day since it's stuck
indexing job took ~3 hours to complete
To Reproduce
Not an error, tasks are just stuck in
processing
stateExpected behavior
35k tasks should be processed swiftly like the previous 76k tasks
Screenshots
Index in
indexing
state for 2 daysTask queue have tasks from yesterday that are still
processing
but finishedtask details
Meilisearch version:
v1.8 with
vectorStore
enabledThe text was updated successfully, but these errors were encountered: