[bug]: Proxy server settings for listeners not working

[koehn]

What happened?

Thanks for building this project! I have high hopes of migrating my existing mail to it.

I have a single instance of stalwart running in a k8s cluster. The cluster is behind haproxy and uses an ingress for HTTP/HTTPS and TLS termination. HaProxy sends SMTP, IMAP, et al directly to stalwart (bypassing ingress-nginx), and sends proxy protocol to those.

When I configure stalwart’s listeners to use proxy protocol thusly (with an app restart to make sure it’s loaded):

server.listener.submission.proxy.override = true
server.listener.submission.proxy.trusted-networks = "10.0.0.0/8"

It ignores the configuration, and when I try to connect, you can see it doesn’t decode the proxy protocol:

$ nc stalwart.koehn.com 587
220 stalwart.koehn.com Stalwart ESMTP at your service
500 5.5.1 Invalid command.
500 5.5.1 Invalid command.
500 5.5.1 Invalid command.
221 2.0.0 Bye.
$ 

Here’s what’s in the log:

2024-08-14T21:59:04Z INFO Network connection started (network.connection-start) listenerId = "submission", protocol = smtp, remoteIp = 10.42.0.1, remotePort = 42669
2024-08-14T21:59:04Z INFO Network connection ended (network.connection-end) listenerId = "submission", protocol = smtp, remoteIp = 10.42.0.1, remotePort = 42669, elapsed = 1ms, protocol = smtp

If I change the server to use proxy protocol by adding this and restarting:

server.proxy.trusted-networks = "10.0.0.0/8"

The connections to most of the listeners work perfectly. But the HTTP listener now expects to use proxy protocol (which is being consumed by ingress-nginx), and there’s nothing I can do to disable proxy protocol on just the HTTP listener (setting server.listener.http.proxy.override = false or changing the trusted-networks has no effect).

So I can either get a functioning HTTP listener, or all the others can work. But not both.

How can we reproduce the problem?

I can reproduce the problem by doing the following steps:

1. Create a new instance using the Docker image.
2. Set the proxy.override and for the non-http services proxy.trusted-networks
3. Connect to the server using proxy protocol
4. Watch it not work

Version

v0.9.x

What database are you using?

RocksDB

What blob storage are you using?

RocksDB

Where is your directory located?

Internal

What operating system are you using?

Docker

Relevant log output

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[mdecimus]

You need to disable the proxy protocol globally and set it on each listener that requires it using proxy.override = true

[koehn]

That’s what I tried. Didn’t work. That’s why I filed a bug.

[mdecimus]

Make sure the server.proxy.trusted-networks directive does not exist on the configuration file or database. You can check use the webadmin if you are not sure whether this setting is on the DB or not.

[koehn]

I used the webadmin to turn off the proxy network. Confirmed it’s not in the config file. Turned of proxy for all listeners Confirmed they’re not in the config file. Turned them all on again and re-added proxy to each in case they’re not in the database. Reloaded, still no joy. Restarted, still no joy.

# egrep 'proxy|networks' etc/config.toml 
server.listener.http.proxy.override = false
server.listener.imap.proxy.override = true
server.listener.imap.proxy.trusted-networks = "10.0.0.0/8"
server.listener.imaptls.proxy.override = true
server.listener.imaptls.proxy.trusted-networks = "10.0.0.0/8"
server.listener.pop3s.proxy.override = false
server.listener.sieve.proxy.override = true
server.listener.sieve.proxy.trusted-networks = "10.0.0.0/8"
server.listener.smtp.proxy.override = true
server.listener.smtp.proxy.trusted-networks = "10.0.0.0/8"
server.listener.submission.proxy.override = true
server.listener.submission.proxy.trusted-networks = "10.0.0.0/8"
server.listener.submissions.proxy.override = true
server.listener.submissions.proxy.trusted-networks = "10.0.0.0/8"
# exit
$ nc -v4 stalwart.koehn.com 587
Connection to home.koehn.com port 587 [tcp/submission] succeeded!
220 stalwart.koehn.com Stalwart ESMTP at your service
500 5.5.1 Invalid command.
500 5.5.1 Invalid command.
500 5.5.1 Invalid command.
221 2.0.0 Bye.

[mdecimus]

Just checked the code and an array is expected, try with:

server.listener.submission.proxy.trusted-networks.0 = "10.0.0.0/8"

[koehn]

That did it. So either the listener code need to handle non-arrays, or the web admin needs to create arrays even when there’s only one element.

[koehn]

I think this should be filed as a bug; should I make a new one?

[mdecimus]

It has been changed already to accept single entries.