Forced dependency on opencl

[hlavaatch]

Fixed linking with the opencl ICD loader means the application will fail to execute at all when OpenCL support is not installed on the machine the application is run.
This makes it impossible to make OpenCL support optional in an application.
Maybe it should be dynamically loaded and fail gracefully instead?

[c0gent]

That would be a great feature. Pull request welcome :)

[dmarcuse]

I've been looking into implementing this using the dlopen crate. Basic testing is promising - I was able to dynamically load OpenCL library at runtime on both Windows and Linux, and confirm that I could map a symbol and call a function from it. However, implementing it is going to be a huge challenge.

The easiest way I can think of would be to have a lazily initialized static field for the OpenCL library container, and rewrite all functions in cl-sys to use this. This would add a bit of overhead to every call, but should maintain public API compatibility.

Any thoughts on this approach before I invest too much time into it?

[hlavaatch]

I believe this can be done without ANY overhead. Filling in imports table from dynamically loaded dll should not be any different from what the loader does.
The tricky part is when to do it.
One option is to require a call to some sort of is_opencl_available() function before any other opencl function is called. Safe but requires cooperation from library users.

Other option would be to boobytrap the opencl imports with an initialization routine which overwrites the imports with the real ones upon first call of any opencl function, then jumps to that function, or returns error safely when not available. Transparent but hacky.

[dmarcuse]

Could you elaborate on how you would hijack in the imports? I've been trying to think of ways to do it without breaking API compatibility.

[hlavaatch]

This is for windows, linux will probably have to be done differently.

1. Get rid of linking the opencl.lib import library. It is unclear where that is anyway.
2. Create a static library that will export all the OpenCL symbols and provides the "thunks" the same way the import library would. The thunks are basically indirect calls through function pointers.
   In an import library, the function pointers are initialized by the loader. In our version, the function pointers would be pre-initialized to point at init stub functions that call the initialization function and then call whatever the initiazliation resulted in.

For each opencl function, we would have:

• Thunk: exported call thru a function pointer (would better be indirect jump, maybe assembly required). This gets exported.
• Init stub, which calls the initialization for all the imports, then calls whatever it resulted in. Function pointer would be initiazlied to this.
• Error stub, a function that can be harmlessly called in the absence of real opencl function. Should return an error code according to thje function signature. Function pointer gets set to this if loading the opencl symbol fails.

Then we would have an initialization function, that would try to LoadLibrary the opencl.dll, and GetProcAddress each export and set the function pointer for each export. If opencl.dll is not found, or does not contain the symbol because it is of an earlier version of opencl, set the function pointer to the error stub.

This way, we end up with at most exactly the same per call overhead as normal import library.

[dmarcuse]

Hmm. If I understand you correctly, this would just be a statically-linked library that exports the OpenCL API, but dynamically loads the actual OpenCL implementation using LoadLibrary/dlopen. This would work, but at that point, how does it differ from just rewriting the cl-sys crate with this behavior?

[hlavaatch]

Yes of course you can do that.

[c0gent]

I'd be happy to merge either (or both) @dmarcuse's dlopen or @hlavaatch's more involved solution (or even a complete ICD loader replacement) as long as they were placed behind a feature flag. I assume this would be as straightforward as using an alternative import to cl-sys (a new cl-sys-dyn or some such library) and adding some conditionally compiled init() calls in various 'entry point' functions (like get_platform_ids, create_context, etc.).

Keep in mind that the extra performance overhead of calling OpenCL functions dynamically is small relative to the cost of OpenCL calls in general due to the latency involved.

It's up to you whether you think it's worth investing time in this @dmarcuse but it would be a nice feature to have. I'd like to help out but I'm stretched pretty thin for the foreseeable future. If you're in a situation where you need to hand write a bunch of FFI code, let me know and I'll try to help out with some of the grunt work.

[dmarcuse]

I'll look into it a little bit more, but right now I'm leaning towards implementing this by rewriting the functions in the cl-sys crate to call functions loaded using the dlopen crate. The performance penalty will be higher, but as @c0gent mentioned, the relative cost is low, and it shouldn't be a very large impact in the first place.

I'm not sure how much will need to be handwritten - I'll need to copy most of the stubs, but I think I can use macros to implement the actual function proxies.

[hlavaatch]

Note that there are other API DLLs in need of similar solution - OpenGL, Vulkan... so having a good generic solution for this would ber great

[hlavaatch]

If you need to save some typing there are lists of the OpenCL functions in the ICD sources at https://github.com/KhronosGroup/OpenCL-ICD-Loader

[dmarcuse]

Thanks! I've been thinking a little bit more about reusability and thought that it might be possible to write a tool to parse the headers and generate Rust code or something, but it may be more work than is necessary.

[kpp]

This makes it impossible to make OpenCL support optional in an application.

@hlavaatch You can using cargo features and adding something like this in your application:

# Cargo.toml:

[dependencies.ocl]
version = "0.19.2"
optional = true

[features]
gpu = ["ocl"]
default = []

# lib.rs:
#[cfg(feature = "gpu")]
mod gpu;

And build your app with cargo build --features gpu.

[dmarcuse]

I was assuming he meant runtime-optional, as in one binary that can enable/disable OpenCL support automatically.

[hlavaatch]

I was assuming he meant runtime-optional, as in one binary that can enable/disable OpenCL support automatically.

I did. :)

[kpp]

Still it is a very specific issue, not a general one. You may compile your binary two times: with gpu and without gpu, then create a third binary which will try to load your binary with gpu and then fallback to a binary without a gpu if the first one fails to load libOpenCL.

[dmarcuse]

I've been working on a new crate, dynamic_ocl, which dynamically loads OpenCL at runtime. A very early (and relatively incomplete) alpha release is available on crates.io if anyone is interested.