SMTP hooks

[soywod]

From #57:

My mail setup is described as above:

* muchsync for syncing mails from my server (or, that is, from my other devices)

* notmuch with neomutt for reading and writing email

* msmtp for sending

I also use msmtp for sending on my current setup. I was about to configure himalaya and stopped at this point because it lacks the option of an external MTA. It's a must for me because I need to pass every email I send through a script before it gets sent (could be done the same way if himalaya provided some internal customization point, a command to run the email over before it's sent).

In fact, the main idea behind is to be able to write emails in markdown. And I love the concept. I see 2 ways of implementing this:

• Via SMTP hooks (as we discussed in the issue), these way we can modify the content before Himalaya sends it
• Via an internal markdown feature

Do you have in mind other applications of hooks? If not then maybe we can consider the internal markdown feature. What do you think @oblitum?

[oblitum]

Hi @soywod, thanks for asking.

An internal markdown feature doesn't sound very flexible, but at the same time sounds good enough for the general public that didn't get into markdown emails yet. Currently my script has got a bunch of details regarding signed messages, html formatting for gmail and outlook, etc, besides going through pandoc for the markdown/html conversion, which allows me to support even LaTeX inside emails (image rendered math). It could also use other original format besides Markdown. So, I'd not like to lose that flexibility.

It doesn't matter to me how modifications would be triggered, if that can be done via SMTP hooks, I'm fine with that, no problem.

[soywod]

So let's go for the hooks. It sounds the most flexible. Maybe later I could integrate a basic markdown support (to have it working "out of the box", for those who just want to write their message in markdown instead of plain text). Thank you for your reply. I opened an issue dedicated to this feature: #178.

[soywod]

I'm changing my mind about such hooks. First I realize that it is not easy at all to set up. Then, I thought maybe it is not that stupid to be able to define your own MTA. I'm already setting up the fact to select your own backend (aka MRA) #295. Could you share a bit more about your msmtp setup? How do you do to transform your markdown content into 2 valid MIME parts text/plain and text/html?

[oblitum]

Hi. I use a script I wrote myself, you can find an explanation here. Notice though that the article as a whole is outdated, I don't use mutt anymore, I'm mostly using meli for quite a while, my setup on the outside has been iterating a lot, but the blog post is still up-to-date on how that script works, it still works the same way, and I really like it. meli has been kinda stagnant and there's some rough edges, hence my interest in competition like himalaya. Before meli I used aerc for a while too, but they lacking proper mail thread support and not syncing all the mail folders concurrently made move out to meli.

[oblitum]

Currently in meli configuration I set an entry like this to invoke MIMEmbellish:

composing.send_mail = 'MIMEmbellish | msmtp --read-recipients --read-envelope-from --passwordeval "pass foobar | head -1"'

[soywod]

Thanks for the precisions. My only concern is how to get the final version of your email (the one sent by msmtp, post treatment) so I can add it to the correct IMAP or Maildir folder? How do you deal with this for now? Let's say you send an email, and you send it with your script, how you get a copy in your sent folder?

[oblitum]

I deal with it two ways. In GMail for example, it doesn't depend on me because they inadvertently store a "sent copy" on their own, I even have to set composing.store_sent_mail = false in meli, which actually has been created due to an issue I opened because I was getting duplicated stores, one by meli, one by the GMail system. This option was created and I enabled it on any GMail accounts. Obviously, GMail will store the final form.

I also have personal mail servers (postfix/dovecot/etc), and for them, what gets stored (by meli) is just my original (before MIMEmbellish) version. I never bothered much on that though, as it has been sufficient.

[soywod]

what gets stored (by meli) is just my original (before MIMEmbellish)

Mmmh, this bothers me a bit. I mean, in theory it should be the same email that has been sent.

I checked a bit your script, so in fact it takes a raw message in parameter, and it modifies it on the fly (adding parts)?

[soywod]

So what is important for you is not really msmtp, but the fact to be able to execute your script before sending the email?

[oblitum]

Yep, yep. As I said formerly, that's all that's relevant for me.

[soywod]

Just tried. It works

Good news 🎉

I got two emails stored after send, one in the inbox of the target, but also one in my inbox, instead of putting it in my "Sent" folder.

I noticed that I create the envelope before calling the hook:

https://github.com/soywod/himalaya/blob/4dc1be25cd7282b1da4f4146dd759ee9a8c885f5/src/smtp/smtp_service.rs#L59-L70

It may be the cause of the issue. I let you know when I push a fix!

[soywod]

I just pushed a fix, could you try again? If it still does not work, could you provide me the raw message (before and after hook execution)?

[oblitum]

Hi @soywod, just tried again, it didn't work. I'll look how to dump the messages before and after, will probably do it from my script, but, I'm not sure how that relates to Himalaya storing sent emails into my inbox instead of sent folder.

[oblitum]

I just tried without the pre-send hook, and the result is the same, so, it's not related to pre-send hook at all, that part is working, but for some other reason, I have this behavior with Himalaya, which is the first time I'm trying out. Please reconsider anything you applied in code attempting to fix this in pre-send code, might not be needed.

[soywod]

Mmmh then it looks like a regression. I cannot reproduce the bug from my side. Could you open a new issue with your himalaya config and the logs (RUST_LOG=debug himalaya write)?

Please reconsider anything you applied in code attempting to fix this in pre-send code, might not be needed.

All is fine, it was needed 🙂

[polyzen]

msmtp/sendmail support is convenient because then one only needs to configure SMTP once, to be used with eg. your MUA and git-send-email.

[soywod]

This is now doable thanks to #177 (reply in thread).

[polyzen]

I mean using msmtp in place of the built-in SMTP support, not as a pre-send hook.

[soywod]

Oh I see. Indeed it could be interesting!

[jarkkojs]

A common use case is to use git send-email to send patches and share the SMTP configuration by using msmtp. I use msmtp with mutt, Evolution and git.

[soywod]

I see. I opened an issue not to forget this point #351. Thanks!