HttpClientHandler client certificates for downstream services

[AlexKuriatnyk]

Hello,

We have a scenario when some services behind the gateway require certificate-based authentication. Is it possible to add client certificate to the downstream call somehow?

[TomPallister]

@AlexKuriatnyk you could do it with delegating handlers docs http://ocelot.readthedocs.io/en/latest/features/delegatinghandlers.html

Or maybe just have a piece of middleware before the Ocelot pipeline.

Let me know if that helps.

[AlexKuriatnyk]

Looks like it is not possible to do this just with delegating handlers. I need to setup client certificate on HttpClientHandler that is used for HttpClient.
Is it possible to add some setting for this or just provide possibility to customize HttpClient creation logic? The only workaround I see for now is to replace entire HttpClientBuilder and HttpClientHttpRequester with DI.

[TomPallister]

@AlexKuriatnyk this should be possible to implement in Ocelot. How do you set the certificate on the HttpClientHandler? If should just be a case of getting the certificate into Ocelot somehow and then passing it through to the HttpClientBuilder to HttpClientHandler.

How does the certificate get passed to the downstream service? Is this like part of the TCP/IP handshake or as a header or something? If it is a header then we might have other options.

[AlexKuriatnyk]

I set it in this way: httpclientHandler.ClientCertificates.Add(certificate);. I believe, it is passed as part of the TCP/IP. I my case, I read it by Subject from certificate storage on machine.
It will be great if it could be possible, for example, to read it on startup and inject into Ocelot with some named key, so then we can use this key in JSON config to indicate that we need to use a certificate for some downstream call.

[TomPallister]

@AlexKuriatnyk ok cool I will take a look at it ASAP!

[yamaritta]

Hello! I have a similar scenario. How I can deal with it?

[secret-agent-B]

Any update on this one? I have a similar issue where I can't reach my downstream host returning "The remote certificate is invalid according to the validation procedure". I have a self-signed cert, by the way.

        public static IWebHostBuilder UseCertificateConfig(this IWebHostBuilder hostBuilder)
        {
            var config = new ConfigurationBuilder()
                   .SetBasePath(Directory.GetCurrentDirectory())
                   .AddEnvironmentVariables()
                   .AddJsonFile("certificate.json", optional: true, reloadOnChange: true)
                   .AddJsonFile($"certificate.{Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT")}.json", optional: true, reloadOnChange: true)
                   .Build();

            hostBuilder
                .UseKestrel()
                .UseConfiguration(config);

            return hostBuilder;
        }

certificate.json

{
    "Kestrel":{
        "Certificates":{
            "Default":{
                "Path":     "localhost.pfx",
                "Password": "mypass"
            }
        }
    }
}

[raman-m]

• SSL Errors

Hope it helps!

[k-u-s]

Hi. I have created two branches that targets this issue. First:
https://github.com/k-u-s/Ocelot/tree/feature/primary_http_client_handler_basic
This just wrapps new HttpClientHandler() in interface that can by then raplaced in DI
And second:
https://github.com/k-u-s/Ocelot/tree/feature/primary_http_client_handler_basic
That allows to register functions that returns HttpClientHandler under key and than based on new Property in HttpHandlerOptions called PrimaryHandlerName specific handler is created.

This way user will be able to change client certs and proxy settings. Could you provide your thoughts about this?

[raman-m]

@k-u-s
Your feature branch doesn't exist!
Moreover, the Ocelot fork doesn't exist!
Can I remove your message from this thread?

[linuxchata]

We tried to adjust HttpClientBuilder class and add client certificate from the initial request to the request to the target URL.

var handler = CreateHandler(context);
handler.ClientCertificates.Add(context.HttpContext.Connection.ClientCertificate);

However, the target API is still not getting the client certificate. The issue might be with the fact that the client certificate from context.HttpContext.Connection.ClientCertificate does not have private key (https://serverfault.com/questions/709812/client-certificate-authentication-with-no-access-to-private-keys).

Certificate on the target API side is read in the following way.

var clientCertificate = await Request.HttpContext.Connection.GetClientCertificateAsync();

Any thought how client certificate can be convey to the target API?

In addition, is there any particular reason to directly create an instance of HttpClientBuilder in the HttpClientHttpRequester? Why DI is not used instead?

[g00fy-]

temporary workaround:

    class X509Hanlder: DelegatingHandler
    {
        private CertificateRepository _certificateRepository;
        public X509Hanlder(CertificateRepository certificateRepository)
        {
            _certificateRepository = certificateRepository;
        }
        protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, System.Threading.CancellationToken cancellationToken)
        {

            var inner = this.InnerHandler;
            while(inner is DelegatingHandler) {
                inner = ((DelegatingHandler)inner).InnerHandler;
            }
            // inner is HttpClientHandler
            if(inner is HttpClientHandler)
            {
                var httpClientHandler = (HttpClientHandler) inner;
                if(httpClientHandler.ClientCertificateOptions != ClientCertificateOption.Automatic)
                {
                    await AddCerificates(httpClientHandler, request);
                }
                
            }

            return await base.SendAsync(request, cancellationToken);
        }

        private async Task AddCerificates(HttpClientHandler httpClientHandler, HttpRequestMessage request)
        {
            X509CertificateCollection ceritificates = await this._certificateRepository.GetAllAsync();
            httpClientHandler.ClientCertificates.AddRange(ceritificates);
            httpClientHandler.ClientCertificateOptions = ClientCertificateOption.Automatic;
        }
    }

[raman-m]

@g00fy-
Why not to create a PR?
Do you have an intention to contribute in 2024?...

[forfront]

Hi,

EDIT: As all I wanted to do was ensure downstream services could only be accessed through an API gateway, I have instead changed the approach to have ocelot add a header with a hash to be verified by the downstream service. The presence of a valid hash will suffice for now.

I am trying to get downstream certificates to work. I simply want to add a certificate to the request received by ocelot and have the downstream service verify the certificate.

I have registered a DelegatingHandler with almost the same code as presented by @g00fy. I see the call to the SendAsync method adding the certificate to httpClientHandler.ClientCertificates. The downstream service is called without issue, but the certificate is not present. i.e. HttpContext.Connection.ClientCertificate returns a null value.

I have created a certificate for secure.local and added imported it into Cert:\LocalMachine\Root
The ocelot api gateway is accessible on: https://secure.local:12000
The downstream service on: https://secure.local:12010

Calls to https://secure.local:12000/U/V1/Testing is setup to reroute to https://secure.local:12010/V1/Testing. This is working, only as mentioned, the certificate added to the request is not being received by the downstream service.

Is there something I am missing?

powershell - used to create self-cert

        [string]$certificateName = "*.secure.local"

        Write-Host "`tProcessing the certificate $($certificateName)..."

	    (Get-ChildItem -path Cert:\LocalMachine\My -Recurse | Where { $_.Subject -eq "CN=$($certificateName)" -and $_.NotAfter -lt  (Get-Date).AddDays(14) }) | Remove-Item
	    (Get-ChildItem -path Cert:\LocalMachine\Root -Recurse | Where { $_.Subject -eq "CN=$($certificateName)" -and $_.NotAfter -lt  (Get-Date).AddDays(14) }) | Remove-Item

        $certificate = (Get-ChildItem -path Cert:\LocalMachine\My -Recurse | Where { $_.Subject -eq "CN=$($certificateName)" })
        $trustedCertificate = (Get-ChildItem -path Cert:\LocalMachine\Root -Recurse | Where { $_.Subject -eq "CN=$($certificateName)" })

        if ($certificate -eq $null)
        {
            Write-Host "`tCreating trusted certificate for $($certificateName) within cert:\LocalMachine\My"
            $certificate = New-SelfSignedCertificate -Subject $($certificateName) -DnsName $($certificateName) -CertStoreLocation cert:\LocalMachine\My -KeyusageProperty All -KeyUsage CertSign, CRLSign, DigitalSignature -KeyExportPolicy Exportable

            Write-Host "`tExporting private/public key trusted certificate for $($certificateName) within cert:\LocalMachine\Root"
            $certificatePwd = ConvertTo-SecureString -String "password-used-to-create-pfx" -Force -AsPlainText
            $certificate | Export-PfxCertificate -FilePath "$($this.RootDir)\Dependencies\secure.local.pfx" -Password $certificatePwd
        }

        if ($trustedCertificate -eq $null)
        {
            Write-Host "`tCreating trusted certificate for $($certificateName) within cert:\LocalMachine\Root"

            $exportedCerFile = Export-Certificate -Cert $certificate -FilePath "$($this.RootDir)\Dependencies\secure.local.cer"

            $exportedCerFile | Import-Certificate -CertStoreLocation Cert:\LocalMachine\Root
        }

ocelot.json

"ReRoutes": [
        {
            "DownstreamPathTemplate": "/{everything}",
            "DownstreamScheme": "https",
            "DownstreamHostAndPorts": [
                {
                    "Host": "localhost",
                    "Port": 12010
                }
            ],
            "UpstreamPathTemplate": "/U/{everything}",
            "UpstreamHttpMethod": [ "Options", "Get", "Post", "Put", "Delete" ],
            "DangerousAcceptAnyServerCertificateValidator": true,
            },
            "DelegatingHandlers": [
                "X509CertificateDelegatingHandler"
            ]
        }

Delegating Handler

    public class X509CertificateDelegatingHandler : DelegatingHandler
    {
        private readonly X509CertificateCollection _certificates = new X509CertificateCollection();

        public X509CertificateDelegatingHandler()
        {
            // testing...
            byte[] rawData = GetFileAsBytes("secure.local.pfx");
            _certificates.Add(new X509Certificate2(rawData, "password-used-to-create-pfx"));
        }

        protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, System.Threading.CancellationToken cancellationToken)
        {
            var inner = InnerHandler;
            while (inner is DelegatingHandler)
            {
                inner = ((DelegatingHandler)inner).InnerHandler;
            }
            // inner is HttpClientHandler
            if (inner is HttpClientHandler httpClientHandler)
            {
                if (httpClientHandler.ClientCertificateOptions != ClientCertificateOption.Automatic)
                {
                    httpClientHandler.ClientCertificates.AddRange(_certificates);
                    httpClientHandler.ClientCertificateOptions = ClientCertificateOption.Automatic;
                }

            }

            return await base.SendAsync(request, cancellationToken);
        }
        private static byte[] GetFileAsBytes(string filePath)
        {
            if (!File.Exists(filePath))
                filePath = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, filePath);
            if (!File.Exists(filePath))
                throw new ApplicationException("Path missing: " + filePath);

            using (FileStream f = new FileStream(filePath, FileMode.Open, FileAccess.Read))
            {
                int size = (int)f.Length;
                byte[] data = new byte[size];
                size = f.Read(data, 0, size);
                f.Close();
                return data;
            }
        }
    }

Kind regards,
Rik

[raman-m]

Hi @forfront !
Your draft solution is based on Delegating Handlers. So, it could be reused in real feature development...

Why not to create a PR?
Do you have an intention to contribute in 2024?...

[ChrisJacobsz97]

@AlexKuriatnyk ok cool I will take a look at it ASAP!

Any update on this issue?

[raman-m]

Seems, No updates by the author. ;)

[bkardisco]

Has there been any status update on this issue? I'm in a similar situation where I want to forward the certificate but no approach so far has worked

[raman-m]

@bkardisco Let us know your user scenario please!

[raman-m]

@AlexKuriatnyk Hello Alex!
Are you still with Ocelot? 🐯

---

We have a scenario when some services behind the gateway require certificate-based authentication.

Could you describe your user scenarios in details please?

---

Is it possible to add client certificate to the downstream call somehow?

Answer: I believe Yes, because ASP.NET provides this feature, but Ocelot not.

Why do you need Ocelot's machine certificates in downstream?
I guess you use some non-standard protocol and scheme... What protocol do you use?

[nikhillkumar]

Hello,

Has there been any update regarding this.
In my case I want to validate the downstream certificate in the ocelot API gateway, so I need to add a custom certificate validator, can anyone provide a solution for this.

Thanks.

[raman-m]

@AlexKuriatnyk on May 12, 2018

We are patient but still wait for your contributions.
Because you are Software Engineer at Microsoft, we believe you have enough expertise to deliver this feature.
Sure we will help you and support during development.

@ggnaegi @RaynaldM FYI

@ggnaegi Seems we need to redesign system core... HttpClient is not our option anymore, right?

[raman-m]

@AlexKuriatnyk
We will be waiting for you here...

[nikhillkumar]

@raman-m, setting DangerousAcceptAnyServerCertificateValidator is something which will cause all certificates to be trusted and hence bypassed, whereas I want to do introduce a custome callback which will do validation based on my custom set of rules, something like ServerCertificateCustomValidationCallback.
Is there something of this sort which I could use ?

[raman-m]

@nikhillkumar
You can search for some ASP.NET docs regarding client certificates and validation on Microsoft Learn portal.
Ocelot doesn't support its own server SSL certificates to send to downstream endpoints. But regular Asp.net app can be configured for this. How to create certificates, how to manage them, how to attach to a route? A lot of questions...
This is absolutely new feature!

I will convert this issue soon to discussion thread, because issue isn't ready for development. And nobody wants to contribute.

[ggnaegi]

@nikhillkumar @raman-m you still have an option, it's adding your own logic here:
https://github.com/ThreeMammals/Ocelot/blob/develop/src/Ocelot/Requester/MessageInvokerPool.cs#L85
You could even go a step further and use DI.

RemoteCertificateValidationCallback = (sender, certificate, chain, sslPolicyErrors) =>
{
... your validation logic here
}

or even better (would help us with a PR)

public interface ICertificateValidator
{
    bool Validate(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors);
}
services.AddSingleton<ICertificateValidator, CustomCertificateValidator>();

[nikhillkumar]

@ggnaegi, thank you so much for pointing in this direction.
As I understand, I will be able to use the above suggested way after a change to the source code of ocelot, so did you mean that you are gonna help with creating a PR ?

[raman-m]

@nikhillkumar

did you mean that you are gonna help with creating a PR ?

Sorry, which PR are you talking about?
Will you work on #357 user scenario (new feature request)?
Or do you want that we will help you to write code for your current tasks of your current employer? )) You want too much, my dear!

Anyway, we are not going to rewrite our generic validator which covers all validation scenarios of all self-signed certificates because it simply returns true without any analysis! We will not change this embedded validator because it covers everything including even your scenario!

Ocelot/src/Ocelot/Requester/MessageInvokerPool.cs

Line 87 in d54836d

RemoteCertificateValidationCallback = (sender, certificate, chain, sslPolicyErrors) => true,

We will not accept and approve any PRs with changes of this line! Moreover it will be removed in future! 👉
The quote from docs:

As a team, we do not consider it as an ideal solution. From one side, the community wants to have an option to work with self-signed certificates. But from other side, currently source code scanners detect 2 serious security vulnerabilities because of this fake validator in 20.0 release. The Ocelot team will rethink this unfortunate situation, and it is highly likely that this feature will at least be redesigned or removed completely.

@ggnaegi FYI !

[ggnaegi]

@raman-m Yes, I just gave a solution to @nikhillkumar, but the feature should be implemented imo.

[raman-m]

the feature should be implemented imo.

By whom? By Mr. Kumar?

[ggnaegi]

@ggnaegi

the feature should be implemented imo.

By whom? By Mr. Kumar?

I think we should first convert this to a discussion and talk about the approach we would like to take. It's an important feature, that is not implemented "out of the box" in other projects (for the known reverse proxies, you will need to compile your own library too).

[raman-m]

@ggnaegi Converted!

It's an important feature, that is not implemented "out of the box" in other projects

Thanks for the info! And now I see by whom it could be implemented 😉
Also FYI there are some discussion threads with the same problem. I don't remember numbers... will link them in future...
But I'm definitely sure there is more productive thread with more details on client certificates.

[ggnaegi]

@raman-m @nikhillkumar Are we mixing concerns here? I realized that the topic was about downstream service authentication using certificates. and, correct me if I'm wrong, @nikhillkumar you are more interested by certificate validation, yes? So, we might have two distincts topics here: 1) Authentication with Certificates, 2) Custom certificates validation.

[raman-m]

What I found now

• Issues with "client certificate" mentioning
• Support for Client certificates for downstream routes #1690 but the discussion failed

[raman-m]

Full list of related issues

• Support for Client certificates for downstream routes #1690
• Consul provider doesn't support client certificate #1426
• Client certificate authentication #1326
• Error making http request #1158
• Not able to forward client certificates and receiving error #1130
• Allow a Route to specify more than one authentication scheme #740
• HttpClientHandler client certificates for downstream services #357

[raman-m]

@ggnaegi Will you have intention and time to implement all these new features?
Currently I see zero candidates who informed me to work on these features.

[ggnaegi]

@raman-m Cool, yes, but for the annual release

[raman-m]

No, it will be not a part of Annual'23 release because it completely independent release branch...
Based on the PR readiness these features will be added to future monthly releases.