-
Notifications
You must be signed in to change notification settings - Fork 63
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Script task as first task is not started #3672
Comments
@matthias-ronge While implementing a naive fix for enabling automatic first workflow tasks (#4853), i began to wonder how this has to be seen in the context of the different process creation modes (#4322). Automatic first workflow steps are probably a prerequisite of fully automated workflows (#3333). On the other hand i do not know what the effect would be if a mass import would trigger a lot of CPU intensive tasks (e.g. generation of image derivatives) by automatically starting a task for every process. |
Our workflow engine is a very old legacy from 10 years of code history. And, we've never did large changes to it, just fixed immediate problems. Even a programmer who authored this a long time ago, should have said, "I'll never touch the workflow engine again". At the moment, the thread that closes the task and opens the next task does the basic work for the task (like creating or removing symbolic links), only a script is started asynchronously in a I think we would need a completely different design in the background to solve this in a clean way. I think there should be a separate task engine where each automatic task is actually represented with a thread (or a few, a fixed number though) that handles those tasks. Actually, I would even see this as a separate program, so that you can install multiple task engines on different servers and they work together. But for that, we would have to reprogram the whole task handling. Or maybe, in the future, make use of a powerful workflow engine and dispose of the self-made one. Related: #4678 |
I see your point. As a (temporary) workaraound maybe it woud help if the workflow editor wouldn't allow such tasks as starting tasks. |
It seems clear that an improper usage of automatic tasks (it might even be multiple chained automatic tasks) might lead to program crashes. So the question is, wether the system admin should decide if it could be risked. "Actually, I would even see this as a separate program, so that you can install multiple task engines on different servers and they work together. " Such a mechanism could probably already be implemented by using ActiveMQ and having multiple workers which take over those tasks. But i do not now if even a the execution of hundreds of small scripts sending a message to active MQ might already be a problem. This could probably not be combined with automatic "internal Kitodo Actions" like generating images since they have to be handled by Kitodo itself. So if somebody wants to trigger multiple first automatic tasks and also generate images in this context, it probaly also has to be done outside of Kitodo. |
Extracted from comment on issue #3279 by @mnscholz:
The text was updated successfully, but these errors were encountered: