Support for sni_hostname extension when using SOCKS proxy.

[Allgot]

For context, please refer to the previous discussion.

A few months back, a new extension named sni_hostname was introduced, and it functions well with connections that don't involve a socks proxy. However, I've discovered that it doesn't work properly when the connection utilizes a socks proxy.

To further investigate the issue, I enabled tracing and performed identical DNS-over-HTTPS (DoH) queries twice, with the only difference: one with a socks proxy and the other without. This comparison revealed an intriguing discrepancy, as shown below. (Note: I've intentionally redacted sensitive information.)

# with socks proxy
socks.connect_tcp.started {'host': '###', 'port': 10000, 'timeout': 5.0}
socks.connect_tcp.complete {'return_value': <httpcore._backends.sync.SyncStream object at 0x7f44566eee60>}
socks.setup_socks5_connection.started {'stream': <httpcore._backends.sync.SyncStream object at 0x7f44566eee60>, 'host': 'cloudflare-dns.com', 'port': 443, 'auth': ###}
socks.setup_socks5_connection.complete {'return_value': <httpcore._backends.sync.SyncStream object at 0x7f44566eee60>}
socks.start_tls.started {'ssl_context': <ssl.SSLContext object at 0x7f445685e840>, 'server_hostname': 'cloudflare-dns.com', 'timeout': 5.0}
socks.start_tls.complete {'return_value': <httpcore._backends.sync.SyncStream object at 0x7f44566eeef0>}
...

# without socks proxy
connection.connect_tcp.started {'host': 'cloudflare-dns.com', 'port': 443, 'local_address': None, 'timeout': 5.0, 'socket_options': None}
connection.connect_tcp.complete {'return_value': <httpcore._backends.sync.SyncStream object at 0x7fb4b90beb90>}
connection.start_tls.started {'ssl_context': <ssl.SSLContext object at 0x7fb4b918b640>, 'server_hostname': 'dns.cloudflare.com', 'timeout': 5.0}
connection.start_tls.complete {'return_value': <httpcore._backends.sync.SyncStream object at 0x7fb4b90beb60>}
...

As you can observe, all other details except the server_hostname in the start_tls event are consistent. Considering that the sni_hostname extension is active, the server_hostname value should be dns.cloudflare.com instead of cloudflare-dns.com. This has led me to suspect whether the newly introduced extension is being properly handled with socks proxy (specifically in the socks_proxy.py file).

Thanks to the detailed logging feature, I was able to quickly identify the source of this issue. Just as I suspected, the socks_proxy.py file wasn't updated correctly. This can be verified through commit #696.

Consequently, I propose making modifications to socks_proxy.py to ensure proper support for the sni_hostname extension.

[karpetrosyan]

Hi, this is very interesting. Can you provide a simple and minimalistic code to reproduce this issue locally?

[Allgot]

If you need, I can also provide the corrected code for socks_proxy.py files! (but it is so simple that you must easily fix :) )

[karpetrosyan]

If we have an issue with the httpcore library, let's give an example just using the functionality that httpcore provides us.

Code

Set up proxy server (defaults to 1080 port)

git clone https://github.com/itsjfx/python-socks5-server
python python-socks5-server/server.py

Client

import httpcore

pool = httpcore.SOCKSProxy("socks5://127.0.0.1:1080")
response = pool.request("GET", 
                        "https://185.199.108.153:443", 
                        headers=[(b'Host', 'www.encode.io')], 
                        extensions={'sni_hostname': 'www.encode.io'})
print(response)

OUTPUT: httpcore.ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid for '185.199.108.153'. (_ssl.c:1123)

But if we use the normal ConnectionPool, this request works properly, am I right?

[Allgot]

Oh sorry. I've intermingled the libraries, HTTPX and httpcore.

Yes, you are right.

The point is that the extension sni_hostname is not handled with socks proxy.

[karpetrosyan]

Thank you so much; that makes perfect sense. Let's double-check to make sure everything is good, and then, if you wish, you can open an issue.

[Allgot]

Alright ;)

Thank you for the quick response!

[tomchristie]

Linking to the issue here... #772