Python exception object is different after pickle.dumps and pickle.loads

@rhettinger

BPO	45112
Nosy	@rhettinger, @iritkatriel
Files	screenshot.png

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-09-06.10:14:34.681>
labels = ['type-bug', '3.9']
title = 'Python exception object is different after pickle.dumps and pickle.loads'
updated_at = <Date 2021-09-06.16:40:31.474>
user = 'https://bugs.python.org/yonghengzero'

bugs.python.org fields:

activity = <Date 2021-09-06.16:40:31.474>
actor = 'rhettinger'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2021-09-06.10:14:34.681>
creator = 'yonghengzero'
dependencies = []
files = ['50262']
hgrepos = []
issue_num = 45112
keywords = []
message_count = 4.0
messages = ['401128', '401135', '401137', '401153']
nosy_count = 3.0
nosy_names = ['rhettinger', 'iritkatriel', 'yonghengzero']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'resolved'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue45112'
versions = ['Python 3.9']

Hi, when I'm trying to serialize/deserialize python exception object through pickle, I found that deserialize result is not the same as the original object...

My python version is 3.9.1, working os: macOS Big Sur 11.4

Here is minimum reproducing code example:

import pickle

class ExcA(Exception):
    def __init__(self, want):
        msg = "missing "
        msg += want
        super().__init__(msg)

ExcA('bb')   # this will output ExcA("missing bb"), which is good
pickle.loads(pickle.dumps(ExcA('bb')))  # this will output ExcA("missing missing bb"), which is different from `ExcA('bb')`

The default __reduce__ method of Exception returns the arg you pass to the Exception constructor:

>>> a = ExcA("a banana")
>>> a.__reduce__()
(<class '__main__.ExcA'>, ('missing a banana',))
>>>

This is then pickled, and when unpickled the arg is passed to the ExcA constructor.

If you want to change this, you can implement __reduce__ on ExcA o return just the part you want it to return.

Pickling customized subclasses can be tricky. The essential details are here: https://docs.python.org/3/library/pickle.html#object.\_\_reduce__

Here's some code to get you started.

class ExcA(Exception):
    def __init__(self, want):
        msg = "missing "
        msg += want
        super().__init__(msg)
    def __reduce__(self):
        return (type(self), self.args, self.args)
    def __setstate__(self, state):
        self.args = stat

It would be nice if exception pickling was more friendly to subclasses. Perhaps, the default should be to store and restore all state including args and a __dict__ if any.

Why do exception return their own class as the callable though? Usually classes return __newobj__ as the callable in order for the instance to be created without calling the constructor (calling the class's __new__) and setting the original state (__dict__) afterwards.

Why does this change of behavior exist for exception instances in the first place?

Navigation Menu

Repository navigation

Python exception object is different after pickle.dumps and pickle.loads #89275

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Python exception object is different after pickle.dumps and pickle.loads #89275

Description

Activity

yonghengzero commented on Sep 6, 2021

iritkatriel commented on Sep 6, 2021

rhettinger commented on Sep 6, 2021

rhettinger commented on Sep 6, 2021

UltimateLobster commented on Aug 24, 2022

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions