Repository navigation
imaplib: incorrect quoting in commands #40038
Description
Activity
anadelonbrin commented
on Mar 16, 2004 anadelonbrinmannequinMannequinAuthorimaplib incorrectly chooses to quote some arguments.
In particular, doing "UID FETCH # BODY.PEEK[]" results
in the BODY.PEEK[] being quoted, and it should not
(according to the RFC), which means the command fails.
This is demonstrated below. It's possible (and
likely) that other UID FETCH arguments are incorrectly
quoted.This occurs with anon cvs python of 16/3/04, and 2.3.3.
Windows XP SP1.I'm happy to provide more info if required, just let me
know. I could try and work up a patch, but it would be
better from someone really familiar with imaplib so
that I don't screw up legitimate quoting.>>> import imaplib >>> i = imaplib.IMAP4("server") >>> i.login("username", "password") ('OK', ['LOGIN Ok.']) >>> i.select() ('OK', ['38']) >>> i.debug = 4 >>> i.uid("FETCH", "96", "BODY") 29:14.23 > GKGP7 UID FETCH 96 BODY 29:14.40 < * 31 FETCH (UID 96 BODY (("text" "plain" ("charset" "iso-8859-1") NIL NIL "quoted-printable" 32 0)("text" "html" ("charset" "iso-8859-1") NIL NIL "quoted-printable" 368 10) "alternative")) 29:14.40 < GKGP7 OK FETCH completed. ('OK', ['31 (UID 96 BODY (("text" "plain" ("charset" "iso-8859-1") NIL NIL "quoted-printable" 32 0)("text" "html" ("charset" "iso-8859-1") NIL NIL "quoted-printable" 368 10) "alternative"))']) >>> i.uid("FETCH", "96", "BODY.PEEK[]") 29:17.04 > GKGP8 UID FETCH 96 "BODY.PEEK[]" 29:17.21 < GKGP8 NO Error in IMAP command received by server. 29:17.21 NO response: Error in IMAP command received by server. ('NO', ['Error in IMAP command received by server.']) >>> i.logout() 29:31.26 > GKGP9 LOGOUT 29:31.42 < * BYE Courier-IMAP server shutting down 29:31.42 BYE response: Courier-IMAP server shutting down 29:31.42 < GKGP9 OK LOGOUT completed ('BYE', ['Courier-IMAP server shutting down']) >>>
anadelonbrin commented
on Mar 16, 2004 anadelonbrinmannequinMannequinAuthorLogged In: YES
user_id=552329Sorry, I missed the bit in the docs that points out that
stuff is always quoted and that using () avoids it. Still,
it does seem that imaplib would be doing it's job better if
it followed correct quoting, rather than always quoting. It
would certainly be easier to use for people familiar with
IMAP, but unfamiliar with imaplib.dmbaggett commented
on May 12, 2009 dmbaggettmannequinMannequinI'm not sure this causes the behavior reported here, but I believe there
really is a bug in imaplib.In particular, it seems wrong to me that this line:
mustquote = re.compile(r"[^\w!#$%&'*+,.:;<=>?^`|~-]")
has \w in it. Should that be \s?
I found this when I noticed that SELECT commands on mailboxes with
spaces in their names failed.dmbaggett commented
on May 12, 2009 dmbaggettmannequinMannequinOK, I missed the initial caret in the regex. The mustquote regex is
listing everything that needn't be quoted, and then negating. I still
think it's wrong, though. According to BNF given in the Formal Syntax
section of RFC 3501, you must must quote atom-specials, which are
defined thus:atom-specials = "(" / ")" / "{" / SP / CTL / list-wildcards /
quoted-specials / resp-specials
list-wildcards = "%" / "*"
quoted-specials = DQUOTE / "\"
resp-specials = "]"So I think this regex should do it:
mustquote = re.compile(r'[()\s%*"]|"{"|"\\"|"\]"')
Changing status to bug.
meatballhat commented
on Aug 4, 2010 meatballhatmannequinMannequinI'm attaching a patch which does exactly what dmbaggett recommended w.r.t. the mustquote regex. All current tests pass, but I'm not sure if the current tests even cover this code (how is coverage measured in the stdlib tests?)
On a related note, the
_checkquotemethod which uses themustquoteregex is dead code in Python 3.2+, AFAICT.8 remaining items
The behavior was weird almost from beginning, when
_checkquote()method was introduced in 8c06221. You can pass arguments either unquoted or quoted, and the heuristic is used to determine if they need quoting. It is ugly design, the API should always take unquoted arguments and quote them if needed. But this was here so long, that there must be much user code which passes quoted values.This got worse in Python 3. The autoquoting was removed in fb5faf0 (except for the password), so users now are forced to pass quoted strings. Since in most cases quoting is not needed, and the documentation does not match the current behavior, users rarely do this, and their code works only until the quoting is needed, then it fails.
We need to restore autoquoting feature. #6395 does this. We should keep the heuristic for compatibility with the code which passes quoted strings. In long long perspective we can deprecate this and always apply proper quoting, but it will be breaking change. We should wait many years until all users get used to passing non-quoted strings. Meanwhile, we can add support of lists/tuples/sets as arguments instead os strings enclosed in parentheses.
What makes this issue more complicated, and what the original report was about, is that different arguments have different quoting rules. For example, "*" should be quoted in the first LIST argument, but not quoted in the second LIST argument. #6395 does not support this. I am working on an alternative solution.
On other hand, we cannot apply auto-quoting even with loosened condition to all arguments, because some arguments that contain spaces should not be quoted. There is an example in RFC 3501:
FETCH 2:4 (FLAGS BODY[HEADER.FIELDS (DATE FROM)])It corresponds to Python code
imap.fetch('2:4', '(FLAGS BODY[HEADER.FIELDS (DATE FROM)])')
Well, in this case
_checkquote()would omit quoting because the argument is enclosed in parentheses, but the following example also satisfies the formal syntax:FETCH 2:4 BODY[HEADER.FIELDS (DATE FROM)]Python code:
imap.fetch('2:4', 'BODY[HEADER.FIELDS (DATE FROM)]')
Now the argument is not enclosed in parentheses, but it contains spaces, so even the most lenient variant of
_checkquote()would quote it.- added 6 commits that reference this issue
on Jul 2, 2026 Argument quoting has been restored in GH-152703, reimplemented per the RFC 3501 grammar: arguments that need quoting are escaped and quoted, flags, sequence sets and list wildcards are left intact, and already quoted arguments are passed through for backward compatibility.
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
Linked PRs