Since merging #8031 in 2.48.0, messages sent while being offline and only sent later when the user gets online, possibly more than a day later, appear not at the end of the chat, but off the screen where they are likely not to be noticed by anyone until #7965 is solved.
This happens because we "render" the message as an (encrypted) MIME blob at the time of sending the message. The Date header (ignoring the fake one in the outer header for encrypted messages) is reflecting the time when the user clicks "send" button, not the time when the message was given to SMTP server.
We want the message to be sorted by the time when it was actually sent. This is similar to how message ordering is working in centralized messengers, where the message is timestamped by the server when the client finally connects to the server and submits it. The message may still be delayed because of server-to-server delays, this is out of scope of the issue and the solution is discussed #7965
Possible solution here is to stop rendering the message into smtp table, but render the message and the Date header from scratch in the SMTP loop right before giving the message to the server. It is not trivial, but possible. Need to keep in mind that:
- Messages may be split into batches of recipients (see
get_max_smtp_rcpt_to)
- Large messages are split into pre- and post- message.
- Messages may be edited and deleted. We already handle deletion by retracting messages from
smtp queue. Edition is handled by sending both the original message and edit even if edit happened offline.
Even with the ordering problem solved we might want the message bubble to display the date of when the message was written ("Send" button clicked), not when the message was sent. Otherwise if the message says "i will be there in 5 minutes" or even "today", but is sent while being offline, the meaning of the message may be lost if it is sent later and the date is shifted forward. We might need another header or sort by the message signature timestamp (for encrypted messages) while keeping Date header as the display date. For unencrypted messages it does not really matter, most likely the messages are read in a non-chat client that makes all unread messages noticeable anyway.
The other option is that we don't solve the problem and treat C2S delays the same way as S2S delays, with #7965
Since merging #8031 in 2.48.0, messages sent while being offline and only sent later when the user gets online, possibly more than a day later, appear not at the end of the chat, but off the screen where they are likely not to be noticed by anyone until #7965 is solved.
This happens because we "render" the message as an (encrypted) MIME blob at the time of sending the message. The
Dateheader (ignoring the fake one in the outer header for encrypted messages) is reflecting the time when the user clicks "send" button, not the time when the message was given to SMTP server.We want the message to be sorted by the time when it was actually sent. This is similar to how message ordering is working in centralized messengers, where the message is timestamped by the server when the client finally connects to the server and submits it. The message may still be delayed because of server-to-server delays, this is out of scope of the issue and the solution is discussed #7965
Possible solution here is to stop rendering the message into
smtptable, but render the message and theDateheader from scratch in the SMTP loop right before giving the message to the server. It is not trivial, but possible. Need to keep in mind that:get_max_smtp_rcpt_to)smtpqueue. Edition is handled by sending both the original message and edit even if edit happened offline.Even with the ordering problem solved we might want the message bubble to display the date of when the message was written ("Send" button clicked), not when the message was sent. Otherwise if the message says "i will be there in 5 minutes" or even "today", but is sent while being offline, the meaning of the message may be lost if it is sent later and the date is shifted forward. We might need another header or sort by the message signature timestamp (for encrypted messages) while keeping Date header as the display date. For unencrypted messages it does not really matter, most likely the messages are read in a non-chat client that makes all unread messages noticeable anyway.
The other option is that we don't solve the problem and treat C2S delays the same way as S2S delays, with #7965