Repository navigation
Defects when parsing email headers aren't raised or propagated to the email object #106144
Description
Bug report
When parsing an email using email.parser.Parser(policy=email.policy.strict), where the email has invalid headers, I expected exceptions to be raised when the email was parsed. Instead, the header objects within the generated EmailMessage record defects, but no exception is raised and the overall EmailMessage has no defects listed.
import email
import email.policy
email_str = '''\
Date: xxx
From: "Bad from header" <bad, value>
'''
parsed_email = email.message_from_string(email_str, policy=email.policy.strict)
# Should never get to this point as the above should already have raised an
# exception.
print(parsed_email['Date'].defects) # Prints "(InvalidDateDefect('Invalid date value or format'),)", expected
print(parsed_email['From'].defects) # Prints "(InvalidHeaderDefect('invalid address in address-list'), ..." , expected
print(parsed_email.defects) # Prints "[]", not expectedYour environment
- Debian 12
- Raspberry Pi 4 (arm64)
- Python 3.11.2 (Debian package 3.11.2-1+b1)
Activity
arhadthedev commentedon Jun 27, 2023
@python/email-team
bitdancer commentedon Jun 27, 2023
You are correct, header defects are theoretically supposed to be copied to the message normally, and be raised in strict mode. It isn't quite as straightforward as that, though :(
Consider the following comment in EmailPolicy:
What this means is that the defects are never even produced until the header is accessed. An email message after parsing is just holding the un-parsed headers in its header list, and knows nothing about what defects it actually has because it hasn't parsed them yet.
So, to make strict work the way it is intended it actually needs to re-implement the header_source_parse method so that it parses the header using the header registry and calls register_defect on any header defects returned.
Clearly, it also needs to be documented that the default EmailPolicy does not gather header defects at message parse time; that defects in headers are only detected when the headers are accessed. And that shouldn't change for the reasons stated in the comment.
One thing we might consider, though, would be to change EmailPolicy's header_fetch_parse method to call register_defect. Then it would be possible to gather all the defects by serializing the message with refold_source set to 'all'. Or maybe we should just document how to create a policy, based on the fixed strict-that-parses-all-headers but with raise_on_defect set to False, for those people who want to gather all the defects in a message. I believe the latter aligns with my original thinking, but clearly I never followed through on this issue.
Oh, and another thing I never followed through on was improving the header parser defects to include more information about where the error occurred, to make the defects more useful when gatheedr into a big list. I also had thoughts about making more error types and/or putting some more structured data into the error object, but none of that ever got further than thoughts in the back of my head.