email.message.Message.add_header behavior differs from Message[header]=v

Bug report

Bug description:

Attempting to copy headers from an existing message to a new (to-be-cleaned copy of the original) message, I iterated over the old message, using something like

for h, v in oldmsg.items():
    newmsg.add_header(h, v)

which works for many of the test-messages in my test Maildir folder, but errors out on certain messages with

    newmsg.add_header(h, v)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/usr/local/lib/python3.13/email/message.py", line 587, in add_header
    self[_name] = SEMISPACE.join(parts)
                  ~~~~~~~~~~~~~~^^^^^^^
TypeError: sequence item 0: expected str instance, Header found

It appears that elements in msg.items() can produce a mixed-content list containing both (str, str) and (str, Header) elements; but add_header doesn't like receiving a Header object as a value, only str.

If I change it to use the .__setitem__() method function:

for h, v in oldmsg.items():
    newmsg[h] = v

it appears to work fine, appending the header that .items() gave me.

According to the .add_header() documentation, it should behave similar to the .__setitem__() method:

This method is similar to __setitem__() except that additional header parameters can be provided as keyword arguments. _name is the header field to add and _value is the primary value for the header.

The fix would involve making add_header accept the same sorts of Header values as __setitem__, or at least align the output of msg.items() with the types that msg.set_header() accepts. If not aligned, at a bare minimum, the documentation should reflect the difference in accepted value types (str vs Header)

This may also have tie-ins with similar Issues here:

(manifested on 3.13 on OpenBSD, but it's standard-library stuff that would likely manifest on any OS)

CPython versions tested on:

3.13

Operating systems tested on:

Other

Linked PRs

gh-151454: Accept Header values in email.message.Message.add_header #151458

the offending header in question was a multi-line X-Ham-Report header. Dropping a breakpoint() in a try/except TypeError block:

(Pdb) print(h)
X-Ham-Report
(Pdb) type(v)
<class 'email.header.Header'>
(Pdb) str(v)
'Spam detection software, running on the system "example.com", has\n identified this incoming email as possible spam.  The original message\n has been attached to this so you can view it (if it isn\'t spam) or label\n similar future email.  If you have any questions, see\n root\\@localhost for details.\n \n Content preview:  Are you still calling people around you in the workplace with\n    worn-out words? Terms such as colleague, employee, boss, and intern are so\n    pass�. Here\'s a fresh supply of words to bring some variety at work. [...]\n    \n \n Content analysis details:   (-1.7 points, 5.0 required)\n \n  pts rule name              description\n ---- ---------------------- --------------------------------------------------\n -0.0 SPF_HELO_PASS          SPF: HELO matches SPF record\n -0.0 SPF_PASS               SPF: sender matches SPF record\n  1.1 DATE_IN_PAST_03_06     Date: is 3 to 6 hours before Received: date\n -2.8 RP_MATCHES_RCVD        Envelope sender domain matches handover relay domain\n  0.0 T_TVD_MIME_EPI         BODY: T_TVD_MIME_EPI\n  0.0 HTML_MESSAGE           BODY: HTML included in message'

(underlying message is from my Word.A.Day mailing-list archive, FWIW)

I'm honestly unsure whether this is deliberate or not. The documentation also says:

In a model generated from bytes, any header values that (in contravention of the RFCs) contain non-ASCII bytes will, when retrieved through this interface, be represented as Header objects with a charset of unknown-8bit.

However, in all examples, values are not header instances. Now, the interface in __setitem__ used for storing the value (whatever its type) is through the policy object and header_store_parse, which becomes a pass-through if we are given a headere instance (because the purpose of the parser is to normalize the inputs and always work with Header instances).

OTOH, I honestly don't know whether add_header should really accept a Header instance because it gives a different semantic of _params, namely it does two different jobs. As such, keyword arguments become irrelevant (and somehow conflicting) if we pass a Header instance as a value and that's for me a no-go. So I would rather explicitly document this.

@bitdancer

cc @bitdancer

Based on the descriptions, I'd expect parity between add_header() and things returned from .items(). If add_header() rejects Header objects, then if feels like .items() should at least consistently return (str, str) pairs rather than a mix of (str, str) and (str, Header). Alternatively, if .items() is going to return both types of tuple, then add_header() should accept both.

I think I actually misread the interface. In the Header description it says:

If you want to include non-ASCII characters in your email headers, say in the Subject or To fields, you should use the Header class and assign the field in the Message object to an instance of Header instead of using a string for the header value

So, Headere instance sare expected to be used when the header value is non-ASCII. I incorrectly assumed that **_params in this case would be for such cases. So, sorry for this and you're right in asking for accepting such objects as otherwise it's impossible to combine header with non-ASCII characters and with specific parameters.

I suspect that, because a Header (should?) likely know how to encode non-7-bit ASCII, the add_header() method might just have to check whether it's a 7-bit ASCII string and do what it already does; and if it's not a 7-bit ASCII string, call some sort of "encode this Header object using the appropriate encoding method" function on the Header and then fall back to the previous case. Or just directly assign the Header value and then have the serialization of the message do that encoding/conversion.

I still wonder what to do with the additional parameters though... it feels weird for them to be ignored (as it is the case in the given PR)

My gut says that both __setitem__() and add_headers() should be the same, i.e., __setitem__() just calls add_headers() with no extra **_params

The Header object doesn't seem well-/officially-documented in detail, but if it contains extra metadata that would translate to **_params info, then using __setitem__(Header) could extract the metadata from the Header into the **_params as part of that process.

The behavior here is a bit wonky for historical reasons: in the transition to python3 we first tried to handle bytes via this unknow-8bit header dodge, but that was a band-aid. Really fixing bytes handling required a whole new API. Returning Header objects from items for dirty 8bit data was quite possibly not the best decision, even at the time, but it was the best I could come up with.

Header is pretty dumb, and somewhat broken. If you want to handle non-ascii, you have to build tuples, call make_header, and then pass the resulting string to add_header, using the params arguments as appropriate if you also have parameters...and you need triples there if you have non-ascii. The new API automates all of this stuff. If you are using a non-compat32 policy you just pass the string with the unicode in it, and it handles any needed encoding (which might be none if policy.utf8=True).

What we need here is a documentation overhaul to emphasize to the right way to use add_header with the new API. Which, to be honest, I have to go back and review before I could write it, though I think the answer is "just pass it a string, which can include unicode". We also need to deprecate compat32 and make 'default' the actual default policy...but that's a job I'm not quite ready to tackle yet, though I'm getting closer ;)

As for the immediate question of whether add_header should accept a Header getitem produces in compat32...maybe? You'd have to disallow the specification of params in that case. It would be a band aid on top of a band aid, though.

In the meantime, if you use a new API policy, it will probably simplify your code...and if it doesn't let me know what the problems are, please.

Right now the PR only covers the simple case: a Header with no params works. If you pass a Header and a param together, you still get an error, just like on main:

>>> msg.add_header('Content-Disposition', hdr, filename='x.ppt')
TypeError: sequence item 0: expected str instance, Header found

So the open question is what that combination should do. Two options:

Keep it narrow: allow a Header only when there are no params, and raise a clear error instead of the confusing one above.
Make the combination work, so a value with non-ASCII characters can carry params too.

I'm fine doing either. And if a docs change is the better fix here, I can close #151458 and write that up instead.

@harjothkhara

@harjothkhara Avoid LLM generated answers. This wastes reviewer's time.

@harjothkhara

@harjothkhara wrote:

Right now the PR only covers the simple case: a Header with no params works. If you pass a Header and a param together, you still get an error

In my original example, I used .add_header() with just a Header object, no additional parameters to conflict, but that's part of what caused the "TypeError: sequence item 0: expected str instance, Header found" error.

I'm fine with the limitation that add_header() reject additional parameters when passing a Header object. And I'm fine with add_header() not taking a Header, but would then expect the documentation to remove the "works like __additem__" portion, or give a stern note detailing that, while __additem__ accepts strings and Header objects, add_header values must be strings, not Header objects.

pythongh-151454: Raise a clear error when add_header() gets a Header …

Fair point. I'll keep this to a concrete decision instead of listing options.

Keeping the narrow fix: add_header accepts a Header when there are no extra params, same as what's in the PR now. Passing params with a Header value still raises a TypeError, but I made it a clear one about mixing a Header with params instead of the confusing "sequence item 0: expected str instance, Header found" message.

This fixes the bug reported here (copying headers from an existing message raises TypeError) without touching the parameterized case. A docs overhaul on the new email API is worth doing but it's a bigger, separate job.

Navigation Menu

Repository navigation

email.message.Message.add_header behavior differs from Message[header]=v #151454

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

email.message.Message.add_header behavior differs from Message[header]=v #151454

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

Gumnos commented on Jun 13, 2026

picnixz commented on Jun 27, 2026

picnixz commented on Jun 27, 2026

Gumnos commented on Jun 27, 2026

picnixz commented on Jun 27, 2026

Gumnos commented on Jun 27, 2026

picnixz commented on Jun 27, 2026

Gumnos commented on Jun 27, 2026

bitdancer commented on Jun 28, 2026

harjothkhara commented on Jul 10, 2026

harjothkhara commented on Aug 2, 2026

picnixz commented on Aug 2, 2026

Gumnos commented on Aug 2, 2026

harjothkhara commented on Aug 15, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions