Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tls: macos load system certificates using security framework. #9539

Merged
merged 1 commit into from
Oct 30, 2024

Conversation

niedbalski
Copy link
Collaborator

Description

This patch should be stacked after

I realised that none of the expected locations for the PEM files was available on my machine. I had some PEM files created by the brew installer but none of them were being detected by fluent-bit.

If the plugin didn't have explicilty set to tls.verify off, I would get a certify verify failed, no matter on which
location I place my CA ring or my pem files.

[2024/10/25 15:46:49] [debug] [calyptia:calyptia.0] created event channels: read=27 write=28
[2024/10/25 15:46:49] [debug] [output:calyptia:calyptia.0] machine_id=73a2e3fa6b6844dd36fcd9970d39537980905f85972aa7f9c210080c55dcf3e2
[2024/10/25 15:46:49] [error] [tls] error: unexpected EOF with reason: certificate verify failed
[2024/10/25 15:46:49] [debug] [upstream] connection #31 failed to cloud-api-staging.calyptia.com:443

Configuration used before

[CUSTOM]
    name calyptia
    api_key xxxx
    fleet_name test-jorge
    calyptia_tls.verify on
    calyptia_host cloud-api.calyptia.com

Therefore, I decided to check if the security framework had the certificates I needed for Letsencrypt, and to my surprise all of those certs were available on the machine security find-certificate -a /System/Library/Keychains/SystemRootCertificates.keychain

Therefore I decided to load certificates from SecTrustSettingsCopyCertificates using the security framework to avoid relying on local certificate paths.

This patch is an implementation of that approach. With this patch applied I get the following output:

[2024/10/25 15:35:04] [debug] [tls] successfully loaded and added certificate 139 to trusted store
[2024/10/25 15:35:04] [debug] [tls] certificate 140 details - subject: /C=BM/O=QuoVadis Limited/CN=QuoVadis Root CA 3, issuer: /C=BM/O=QuoVadis Limited/CN=QuoVadis Root CA 3
[2024/10/25 15:35:04] [debug] [tls] successfully loaded and added certificate 140 to trusted store
[2024/10/25 15:35:04] [debug] [tls] certificate 141 details - subject: /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root G2, issuer: /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root G2
[2024/10/25 15:35:04] [debug] [tls] successfully loaded and added certificate 141 to trusted store
[2024/10/25 15:35:04] [debug] [tls] certificate 142 details - subject: /C=GB/O=Sectigo Limited/CN=Sectigo Public Time Stamping Root E46, issuer: /C=GB/O=Sectigo Limited/CN=Sectigo Public Time Stamping Root E46
[2024/10/25 15:35:04] [debug] [tls] successfully loaded and added certificate 142 to trusted store
[2024/10/25 15:35:04] [debug] [tls] certificate 143 details - subject: /C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies, Inc./CN=Starfield Root Certificate Authority - G2, issuer: /C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies, Inc./CN=Starfield Root Certificate Authority - G2
[2024/10/25 15:35:04] [debug] [tls] successfully loaded and added certificate 143 to trusted store
[2024/10/25 15:35:04] [debug] [tls] certificate 144 details - subject: /C=GR/L=Athens/O=Hellenic Academic and Research Institutions Cert. Authority/CN=Hellenic Academic and Research Institutions ECC RootCA 2015, issuer: /C=GR/L=Athens/O=Hellenic Academic and Research Institutions Cert. Authority/CN=Hellenic Academic and Research Institutions ECC RootCA 2015
[2024/10/25 15:35:04] [debug] [tls] successfully loaded and added certificate 144 to trusted store
[2024/10/25 15:35:04] [debug] [tls] certificate 145 details - subject: /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA, issuer: /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA
[2024/10/25 15:35:04] [debug] [tls] successfully loaded and added certificate 145 to trusted store
[2024/10/25 15:35:04] [debug] [tls] certificate 146 details - subject: /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root G3, issuer: /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root G3
[2024/10/25 15:35:04] [debug] [tls] successfully loaded and added certificate 146 to trusted store
[2024/10/25 15:35:04] [debug] [tls] certificate 147 details - subject: /C=US/O=Google Trust Services LLC/CN=GTS Root R1, issuer: /C=US/O=Google Trust Services LLC/CN=GTS Root R1
[2024/10/25 15:35:04] [debug] [tls] successfully loaded and added certificate 147 to trusted store
[2024/10/25 15:35:04] [debug] [tls] certificate 148 details - subject: /C=US/O=Certainly/CN=Certainly Root E1, issuer: /C=US/O=Certainly/CN=Certainly Root E1
[2024/10/25 15:35:04] [debug] [tls] successfully loaded and added certificate 148 to trusted store
[2024/10/25 15:35:04] [debug] [tls] certificate 149 details - subject: /C=BM/O=QuoVadis Limited/CN=QuoVadis Root CA 2, issuer: /C=BM/O=QuoVadis Limited/CN=QuoVadis Root CA 2
[2024/10/25 15:35:04] [debug] [tls] successfully loaded and added certificate 149 to trusted store
[2024/10/25 15:35:04] [debug] [tls] certificate 150 details - subject: /C=PL/O=Unizeto Sp. z o.o./CN=Certum CA, issuer: /C=PL/O=Unizeto Sp. z o.o./CN=Certum CA
[2024/10/25 15:35:04] [debug] [tls] successfully loaded and added certificate 150 to trusted store
[2024/10/25 15:35:04] [debug] [tls] certificate 151 details - subject: /C=US/ST=New Jersey/L=Jersey City/O=The USERTRUST Network/CN=USERTrust RSA Certification Authority, issuer: /C=US/ST=New Jersey/L=Jersey City/O=The USERTRUST Network/CN=USERTrust RSA Certification Authority
[2024/10/25 15:35:04] [debug] [tls] successfully loaded and added certificate 151 to trusted store
[2024/10/25 15:35:04] [debug] [tls] certificate 152 details - subject: /C=US/O=Microsoft Corporation/CN=Microsoft RSA Root Certificate Authority 2017, issuer: /C=US/O=Microsoft Corporation/CN=Microsoft RSA Root Certificate Authority 2017
[2024/10/25 15:35:04] [debug] [tls] successfully loaded and added certificate 152 to trusted store
**[2024/10/25 15:35:04] [debug] [tls] finished loading keychain certificates, total loaded: 153**
[2024/10/25 15:35:04] [debug] [output:calyptia:calyptia.0] machine_id=73a2e3fa6b6844dd36fcd9970d39537980905f85972aa7f9c210080c55dcf3e2
[2024/10/25 15:35:04] [debug] [tls] connection and handshake ok
[2024/10/25 15:35:04] [debug] [http_client] not using http_proxy for header
[2024/10/25 15:35:04] [ info] [output:calyptia:calyptia.0] connected to Calyptia, agent_id='07622b68-bffc-4169-b38c-f01d3798885f'
[2024/10/25 15:35:04] [ info] [sp] stream processor started
[2024/10/25 15:35:10] [debug] [task] created direct task=0x11b004900 id=0 OK
[2024/10/25 15:35:10] [debug] [tls] connection and handshake ok
[2024/10/25 15:35:10] [debug] [upstream] KA connection #34 to cloud-api-staging.calyptia.com:443 is connected
[2024/10/25 15:35:10] [debug] [http_client] not using http_proxy for header
[2024/10/25 15:35:10] [debug] [output:calyptia:calyptia.0] metrics delivered OK
[2024/10/25 15:35:10] [debug] [upstream] KA connection #34 to cloud-api-staging.calyptia.com:443 is now available
[2024/10/25 15:35:10] [debug] [out flush] cb_destroy coro_id=0
[2024/10/25 15:35:10] [debug] [task] destroy task=0x11b004900 (task_id=0)

Testing

  • Example configuration file for the change
  • Debug log output from testing the change

Documentation

  • Documentation required for this feature

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Copy link
Contributor

@cosmo0920 cosmo0920 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically, the implementation should be good. I added a small concerns for coding style, message style, and the counter to count already loaded certificates.

src/tls/openssl.c Outdated Show resolved Hide resolved
src/tls/openssl.c Outdated Show resolved Hide resolved
src/tls/openssl.c Outdated Show resolved Hide resolved
src/tls/openssl.c Outdated Show resolved Hide resolved
Load the system certicates using the security framework instead
of a depending on a local pem file.

Signed-off-by: Jorge Niedbalski <[email protected]>
Copy link
Contributor

@cosmo0920 cosmo0920 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@niedbalski niedbalski merged commit 3da359e into master Oct 30, 2024
85 of 86 checks passed
@niedbalski niedbalski deleted the niedbalski-tls-macos branch October 30, 2024 10:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs-required ok-package-test Run PR packaging tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants