Skip to content

Wrong Patch Generated #138

Description

@rmorshea

The following sample of code ought to work, but instead it raises an error:

from jsonpatch import apply_patch, make_patch

old = [
    {"x": ["a", {"y": ["b"]}], "z": "a"},
    {"x": ["c", {"d": ["d"]}], "z": "c"},
    {},
]

new = [
    {"x": ["c", {"y": ["d"]}], "z": "c"},
    {},
]

patch = make_patch(old, new)

assert apply_patch(old, patch) == new
Traceback (most recent call last):
  File "/venv/lib/python3.9/site-packages/jsonpointer.py", line 275, in walk
    return doc[part]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/temp.py", line 18, in <module>
    assert apply_patch(old, patch) == new
  File "/venv/lib/python3.9/site-packages/jsonpatch.py", line 151, in apply_patch
    return patch.apply(doc, in_place)
  File "/venv/lib/python3.9/site-packages/jsonpatch.py", line 669, in apply
    obj = operation.apply(obj)
  File "/venv/lib/python3.9/site-packages/jsonpatch.py", line 324, in apply
    subobj, part = self.pointer.to_last(obj)
  File "/venv/lib/python3.9/site-packages/jsonpointer.py", line 194, in to_last
    doc = self.walk(doc, part)
  File "/venv/lib/python3.9/site-packages/jsonpointer.py", line 278, in walk
    raise JsonPointerException("index '%s' is out of bounds" % (part, ))
jsonpointer.JsonPointerException: index '1' is out of bounds

The generated patch is:

[
    {"op": "remove", "path": "/0/x/0"},
    {"op": "replace", "path": "/0/x/1/y/0", "value": "d"},
    {"op": "replace", "path": "/0/z", "value": "c"},
    {"op": "remove", "path": "/1/x"},
    {"op": "move", "from": "/1/z", "path": "/0/x/0"},
    {"op": "remove", "path": "/2"},
]

Which is clearly the result of the error above.

Activity

  1. rmorshea commented on Dec 24, 2021

    @rmorshea
    Author

    It looks like swapping these two items fixes the patch:

    {"op": "remove", "path": "/0/x/0"}
    {"op": "replace", "path": "/0/x/1/y/0", "value": "d"}

    So that the remove operation happens second:

    {"op": "replace", "path": "/0/x/1/y/0", "value": "d"}
    {"op": "remove", "path": "/0/x/0"}
  2. rmorshea commented on Dec 24, 2021

    @rmorshea
    Author

    Removing this section of code (which I assume is some optimization heuristic) fixes the assertion: https://github.com/stefankoegl/python-json-patch/blob/master/jsonpatch.py#L828-L851

  3. rmorshea commented on Jan 4, 2022

    @rmorshea
    Author

    For those who need to work around this until a patch is released, the following code will fix this issue (though probably with a hit to performance):

    from jsonpatch import _ST_REMOVE
    from jsonpatch import DiffBuilder as _DiffBuilder
    from jsonpatch import JsonPatch as _JsonPatch
    from jsonpatch import JsonPointer, RemoveOperation, _path_join
    
    
    def apply_patch(doc, patch, in_place=False, pointer_cls=JsonPointer):
        if isinstance(patch, (str, bytes)):
            patch = JsonPatch.from_string(patch, pointer_cls=pointer_cls)
        else:
            patch = JsonPatch(patch, pointer_cls=pointer_cls)
        return patch.apply(doc, in_place)
    
    
    def make_patch(src, dst, pointer_cls=JsonPointer):
        return JsonPatch.from_diff(src, dst, pointer_cls=pointer_cls)
    
    
    class JsonPatch(_JsonPatch):
        @classmethod
        def from_diff(
            cls,
            src,
            dst,
            optimization=True,
            dumps=None,
            pointer_cls=JsonPointer,
        ):
            json_dumper = dumps or cls.json_dumper
            builder = DiffBuilder(src, dst, json_dumper, pointer_cls=pointer_cls)
            builder._compare_values("", None, src, dst)
            ops = list(builder.execute())
            return cls(ops, pointer_cls=pointer_cls)
    
    
    class DiffBuilder(_DiffBuilder):
        def _item_removed(self, path, key, item):
            new_op = RemoveOperation(
                {
                    "op": "remove",
                    "path": _path_join(path, key),
                },
                pointer_cls=self.pointer_cls,
            )
            new_index = self.insert(new_op)
            self.store_index(item, new_index, _ST_REMOVE)
  4. changed the title [-]Invalid Patch Generated[/-] [+]Wrong Patch Generated[/+] on Mar 18, 2022
  5. rmorshea commented on Jun 26, 2022

    @rmorshea
    Author

    @stefankoegl any ideas on what's going wrong here? I've narrowed down the issue about as much as I can without having to put in a bunch more time to figure out the underlying issue.

  6. rmorshea commented on Aug 12, 2022

    @rmorshea
    Author

    @stefankoegl do you have any hints you could give me on how to fix this?

    It turns out that my patch hasn't complete fixed this bug and I'm in desperate need of a solution.

  7. stefankoegl commented on Aug 12, 2022

    @stefankoegl
    Owner

    Unfortunately, I don't have anything I can suggest. I haven't been actively working on the project for quite some time now, so I'm not up-to-speed at the moment.

  8. rmorshea commented on Aug 13, 2022

    @rmorshea
    Author

    Turns out my hacked fix still works. The new error I was experiencing was caused by something else.

  9. ckarnell commented on Sep 5, 2026

    @ckarnell

    While looking into this issue I hit a related case in the same area that fails silently instead of raising: make_patch produces a patch that applies without any error but yields the wrong document. On jsonpatch 1.33:

    import jsonpatch
    
    src = [None, True, {}]
    dst = [[{"1": 1, "a": "s"}], {"": {"a": None, "1": 0}}, {"a": True}]
    
    patch = jsonpatch.make_patch(src, dst)
    # [{'op': 'replace', 'path': '/0', 'value': [{'1': 1, 'a': 's'}]},
    #  {'op': 'add', 'path': '/1', 'value': {'': {'a': None, '1': 0}}},
    #  {'op': 'move', 'from': '/1', 'path': '/2/a'}]
    
    result = patch.apply(src)
    # result   == [[{'1': 1, 'a': 's'}], True, {'a': {'': {'a': None, '1': 0}}}]
    # expected == dst
    assert result == dst   # fails, and no exception was raised

    The generated move from /1 (right after the add at /1) lands in the wrong place, so the applied result differs from dst with no error surfaced. It appears to come from the same move-optimization path as the crash in this issue, just showing up as a wrong document rather than an exception.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions