SMTPUTF8 support issue

[gegorov2030]

In my project involving PowerMTA, I've identified a critical issue that warrants attention. PowerMTA consistently signals SMTPUTF8 support availability upon connection. This standard behavior leads to complications: PowerMTA enforces SMTPUTF8 support requests when interfacing with subsequent ESP nodes, such as Yahoo and AOL, which do not support SMTPUTF8. This mismatch results in PowerMTA rejecting emails without verifying the actual need for SMTPUTF8 support in each case.

The core of the challenge lies in adapting the Client.Mail(from) method, which is constrained by its reliance on private methods, thus limiting flexibility in behavior modification. The method's design, specifically the construction of the command string (cmdStr), is rigid and doesn't accommodate changes in SMTPUTF8 handling.

My thoughts on possible solutions are:

• Adding a parameter to the method seems inelegant and not very reusable.
• Incorporating options, similar to parameters, also feels impractical for this specific issue.
• Extracting the command string generation into a separate strategy with a default implementation. This approach seems the most feasible, although I doubt its necessity beyond my specific case.

An additional consideration is implementing a default mechanism to evaluate the necessity of SMTPUTF8, based on the email address. If an address lacks specific UTF8 characters, the SMTPUTF8 extension requirement could potentially be bypassed.

Thank you, for any suggestions

[wneessen]

Hi @gegorov2030,

that's an interesting issue. I'll have to do some investigation and testing to see what would be the best solution. The good thing is that we forked net/smtp from the Go stdlib, so we can actually influence this behaviour. My first gut feeling is, to detect any UTF-8 in the mail address and then decide if to trigger the SMTPUTF8 extension. From what I can see, the Postfix mail server follows a similar strategy: https://www.postfix.org/SMTPUTF8_README.html#using

Regarding PowerMTA, I've never heard of that product before. Is there a way for me to install that on my DEV machine, so I can perform some testing myself?

I'll invest some time tomorrow and see what I can come up with.

[gegorov2030]

Unfortunately, as far as I know, PowerMTA (PMTA) does not provide a testing environment for personal or development use. However, I have access to a corporate instance of PMTA at my disposal. If you need to conduct any tests, I can assist with that.

[wneessen]

Reading further into RFC 6531, the SMTPUTF8 extension does not only focus on envelope addresses like the FROM and TO addresses, but it can also add UTF-8 headers like Message-ID and so on. So to detect if we actually need SMTPUTF8, we would probably need to scan all headers for any non-ASCII character and then decide on that. Additionally the smtpclient.Mail() method is public, so we can't simply change its signature without breaking other code.

So my idea for a simple workaround is, since this seem to be isolated to PowerMTA, that maybe we just add a bool flag to the smtp client skipSMTPUTF8 or similar and allow the Client to be created with an additional option that would then set this flag to true. If smtpclient.Mail() sees that the flag is set, it would not issue the SMTPUTF8 header with the MAIL FROM even if the server supports it. This of course means, that if the server supports it and the headers contain UTF8, the receiving server would likely reject it.

Alternatively we actually build some header scanner that makes sure no UTF8 is within the headers and then automagically triggers that flag, so that we only issue the extension if support and actually needed.

[gegorov2030]

Thank you for the additional research you did. There is no issue for me to use the option in my personal fork. I don’t see the need to add this case for everyone while cluttering the API with additional options. I like the idea of scanning all message headers in Client.Mail, but yes, this would require losing backward compatibility. Perhaps it's worth delaying this change until other big changes happen?