Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize the performance of series migration #3300

Closed
matthias-ronge opened this issue Mar 18, 2020 · 2 comments
Closed

Optimize the performance of series migration #3300

matthias-ronge opened this issue Mar 18, 2020 · 2 comments

Comments

@matthias-ronge
Copy link
Collaborator

The migration function for series and multi-volume works is very slow. It processes an average of 1000 processes per hour. Just checking whether a process folder contains the anchor file (and if not, skipping it) takes a whole second. This is because services are used through the service loader, it is all very complex. There was also the impression that the existence of files was repeatedly checked many times.

Goal: Review functionality and improve performance. Do more things on the spot, remove everything that is unnecessary to increase processing speed.

@andre-hohmann
Copy link
Collaborator

@matthias-ronge: Does the problem still exists?

The SLUB Dresden faced the same problem, but it works now.

@matthias-ronge
Copy link
Collaborator Author

By now, I had no chance to test it again.

Performance issues typically involve various factors, so judging this is always subjective. If you can confirm that a recent hierarchy migration of the SLUB data is going ahead in a timely manner, we can also close this and, if it turns out badly again, open it again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants