-
Notifications
You must be signed in to change notification settings - Fork 80
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
md5_mb (multibuffer) using isa-l_crypto #6037
Conversation
@@ -0,0 +1,34 @@ | |||
FROM centos:8 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- What is the use case for this Docker file?
- Can we add an entry in the Makefile?
- We can Update the https://github.com/noobaa/noobaa-core/wiki/Building-and-Testing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am ignoring this for now. Will get back to it later.
13923c0
to
6baa16a
Compare
Don't forget
in object_server |
@nimrod-becker no need anymore, monkey-patching works for that as well. |
TypeError: Cannot read property 'unwrap' of undefined Co-Authored-By: liranmauda <[email protected]>
Explain the changes
All mains now import
src/util/fips
module on startup which auto detects if running in fips mode based on/proc/crypto/fips
. This can also be forced using the FIPS=1 env.In fips mode in order to allow md5 for non cryptographic use cases needed for the S3 protocol, we use md5_mb from isa-l_crypto. Since crypto.createHash is called from our code and also node modules, we monkey-patch the crypto library and replace the createHash('md5'), and also use it natively in the upload pipeline (ChunkSplitter).
NOTE: Using md5_mb with a single stream provides lower performance compared to openssl MD5. See this issue on single buffer performance - intel/isa-l_crypto#45. We try to leverage the multibuffer capability by sharing a thread_local ctx mgr and deferring the flushing of contexes so that if multiple streams are processed concurrently it will be flushed together. The single CPU core performance gain can be very significant - see the performance report here - https://github.com/intel/isa-l/wiki/Documentation.
Issues: Fixed #xxx / Gap #xxx
Testing Instructions: