Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minimize array creation #71

Open
Korijn opened this issue May 1, 2023 · 2 comments
Open

Minimize array creation #71

Korijn opened this issue May 1, 2023 · 2 comments

Comments

@Korijn
Copy link
Contributor

Korijn commented May 1, 2023

As an extension to pygfx/pygfx#520 we can optimize pylinalg functions by minimizing (temporary) array creation, reducing time spent in memory allocation and garbage collection.

This will have bigger impact when the functions are used on arrays of inputs.

@FirefoxMetzger
Copy link
Contributor

An alternative to changing everything to in-place operations could be to introduce numba or cython.

This would mean that kernels get compiled and that we have to ship platform-specific wheels, but it would take care of optimizing buffer reuse, reducing the overhead from crossing the C-Python boundary between numpy calls, and can optionally release the GIL during kernel execution. The last point is interesting, because - if done right - it synergizes with asyncio and python's multi-threading primitives to allow multi-threaded computation that can run in parallel to the interpreter (i.e., you can sidestep the GIL with it).

@Korijn
Copy link
Contributor Author

Korijn commented May 1, 2023

Let's cross out Numba right away, if we go that path it will be Cython.

(Numba as a runtime dependency is not an option due to the compiler requirement, and their AOT story is very poorly supported)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants