Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lazy converter instantiation performance gotcha #62

Open
DerGuteMoritz opened this issue Sep 12, 2022 · 1 comment
Open

Lazy converter instantiation performance gotcha #62

DerGuteMoritz opened this issue Sep 12, 2022 · 1 comment

Comments

@DerGuteMoritz
Copy link
Collaborator

Converters are instantiated (and then memoized) lazily on first use. Since this is a rather heavy operation, it can influence timing-sensitive code as recently observed in Aleph's test suite. This is bit of a gotcha which was even brought up in a past issue already. That resulted in the introduction of a precache-conversions API. However, it later got removed again without further explanation. In any case, the fact that this issue came up again indicates that such an API would still be desirable.

@KingMob
Copy link
Collaborator

KingMob commented Sep 13, 2022

Before we start talking about an API, let's define the problem more clearly.

We know the late binding can mess with tests and other initial conversions.

We don't know the impact of just doing it eagerly. Based on the test, we know it's more than a few hundred ms, but that's not a big deal at startup time for a long-running process. Can you time the eager setup of all conversions? How bad is it really? If it's not too bad, we should just simplify and switch. It's totally possible the initial setup isn't as painful as when byte-streams was created.

If it's considerable, we do have another option. We could split the def-conversions into different namespaces, and users would require only the ones they need. Not sure that's much better, though.

In general, I'd prefer to optimize for predictability over startup time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants