Bad UTF-8 "To" header encoding? · Issue #369 · anymail/django-anymail (original) (raw)

Hi!

I'm getting errors when trying to send messages when the "to" header has non-ASCII chars.
The problem seems to happen when all these conditions are true at the same time:

  1. the readable name part of the recipient is big (e.g. "A very long and big name for this recipient" to@example.com)
  2. there is at least one non-ASCII char
  3. there is at least one "special char" (only seems to happen with comma or parenthesis)
  4. the special char isn't close to or between no-ASCII chars

I will use the related test to better explain:

Description Input Output Error?
Current test "Người nhận" to@example.com To: =?utf-8?b?TmfGsOG7nWkgbmjhuq1u?= to@example.com No
Long "Người nhận a very very long name" to@example.com To: =?utf-8?b?TmfGsOG7nWkgbmjhuq1u?= a very very long name to@example.com No
With comma "Người nhận a very very long, name" to@example.com To: =?utf-8?b?TmfGsOG7nWkgbmjhuq1u?= a very very long, name to@example.com YES
Comma between non-ASCII "Người nhận a very very long, náme" to@example.com To: =?utf-8?b?TmfGsOG7nWkgbmjhuq1uIGEgdmVyeSB2ZXJ5IGxvbmcsIG7DoW1l?=\n to@example.com No
Comma near non-ASCII "Người nhận, name" to@example.com To: =?utf-8?b?TmfGsOG7nWkgbmjhuq1uLCBuYW1l?= to@example.com No

So, if we have a UTF-8 encoded part and a special char, the special char must also be encoded, but currently this is only happening if the special char is close or between non-ASCII chars.

Commenting this line seems to encode the entire name in these cases, solving the problem. But I don't know if this has other unwanted consequences.

Example traceback:

File ~/.local/lib/python3.11/site-packages/anymail/backends/amazon_ses.py:78, in EmailBackend._send(self, message)
     76 def _send(self, message):
     77     if self.client:
---> 78         return super()._send(message)
     79     elif self.fail_silently:
     80         # (Probably missing boto3 credentials in open().)
     81         return False

File ~/.local/lib/python3.11/site-packages/anymail/backends/base.py:147, in AnymailBaseBackend._send(self, message)
    144     return False
    146 payload = self.build_message_payload(message, self.send_defaults)
--> 147 response = self.post_to_esp(payload, message)
    148 message.anymail_status.esp_response = response
    150 recipient_status = self.parse_recipient_status(response, payload, message)

File ~/.local/lib/python3.11/site-packages/anymail/backends/amazon_ses.py:114, in EmailBackend.post_to_esp(self, payload, message)
    110     response = client_send_api(**payload.params)
    111 except BOTO_BASE_ERRORS as err:
    112     # ClientError has a response attr with parsed json error response
    113     # (other errors don't)
--> 114     raise AnymailAPIError(
    115         str(err),
    116         backend=self,
    117         email_message=message,
    118         payload=payload,
    119         response=getattr(err, "response", None),
    120     ) from err
    121 return response

AnymailAPIError: An error occurred (BadRequestException) when calling the SendEmail operation: Local address contains control or whitespace
botocore.errorfactory.BadRequestException: An error occurred (BadRequestException) when calling the SendEmail operation: Local address contains control or whitespace