gc.get_referrers() is inherently dangerous #39117

Activity

1. arigo commented on Aug 23, 2003

   

   arigomannequin

   on Aug 23, 2003

   MannequinAuthor

   More actions

   gc.get_referrers() can be used to crash any Python
   interpreter because it allows the user to obtain
   objects which are still under construction.

   Here is an example where an iterator can use it to
   obtain a tuple while it is still being populated by the
   'tuple' built-in function. The following example
   triggers a SystemError, but as the tuple 't' is at the
   moment still full of null values it can easily generate
   segfaults instead.

   from gc import get_referrers
   
   def iter():
       tag = object()
       yield tag   # 'tag' gets stored in the result tuple
       lst = [x for x in get_referrers(tag)
              if isinstance(x, tuple)]
       t = lst[0]  # this *is* the result tuple
       print t     # full of nulls !
   
   tuple(iter())

   Unless someone has more ideas than me as to how to
   prevent this problem, I'd suggest that
   gc.get_referrers() should be deemed 'officially
   dangerous' in the docs.
2. arigo

   closed this as completedon Aug 23, 2003
3. arigo

   added

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   on Aug 23, 2003
4. arigo

   closed this as completedon Aug 23, 2003
5. arigo

   added

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   on Aug 23, 2003
6. ods commented on Aug 28, 2003

   

   odsmannequin

   on Aug 28, 2003

   Mannequin

   More actions

   Logged In: YES
   user_id=63454

   I guess it's dangerous to make object that is not properly
   initialized reachable from other code. Even if it's reachable
   with get_referrers() only. There is no danger in
   get_referrers() itself.
7. arigo commented on Aug 28, 2003

   

   arigomannequin

   on Aug 28, 2003

   MannequinAuthor

   More actions

   Logged In: YES
   user_id=4771

   But it would be very difficult to fix the code base to avoid
   the problem. The 'tuple' constructor was only an example; it
   is actually a quite common pattern everywhere in the C code
   base of both the core and extension modules. Expecting an
   object not to be seen before you first hand it out is
   extremely common, and get_referrers() breaks that
   assumption. Hence the claim that the problem really lies in
   get_referrers().
8. loewis commented on Aug 31, 2003

   

   loewismannequin

   on Aug 31, 2003

   Mannequin

   More actions

   Logged In: YES
   user_id=21627

   I see no harm in your example. Even though the tuple has
   NULLs in it, is in a stable state. The lesson learned is
   that an object should become gc-tracked only if it is in a
   stable state, as gc-tracking means to expose references to
   the object. This is true independent of get_referrers, as
   gc-tracking means that tp_traverse might be called for the
   object, so it *has* to be in a stable state.

   I fail to see how the example "crashes" the Python
   interpreter. I causes a SystemError when the tuple is
   resized, that's all. There are many ways to cause a
   SystemError, including

   raise SystemError

   I recommend not to declare a function as "dangerous" in the
   docs. Instead, the actual problem should be explained, both
   in the GC part of the C API, and in gc module docs (for both
   get_referrers, and get_objects).
9. tim-one commented on Aug 31, 2003

   

   tim-one

   on Aug 31, 2003

   Member

   More actions

   Logged In: YES
   user_id=31435

   Martin, it's easy to transform the example into one that
   crashes. For example, adding "print t[3]" as the last line of
   the iter() function causes it to die with a segfault on my box.
   Virtually anything that fetches a NULL from the tuple will try
   to Py_INCREF it (that's all "t[3]" does), and that's an instant
   NULL-pointer dereferencing death.

   That said, it would be more helpful <wink> if Armin submitted
   a doc patch. The introspective features of the gc module are
   intended to be used by consenting adults facing hard
   debugging problems, and I don't care if they can be tricked
   into blowing up. I agree the docs should point out the
   possibility, though.
10. arigo commented on Sep 2, 2003

    

    arigomannequin

    on Sep 2, 2003

    MannequinAuthor

    More actions

    Logged In: YES
    user_id=4771

    Here is the doc patch (attached).

    A clean solution to this problem would involve delaying GC
    registration, which I think would imply changes in the C
    API. It is probably not worth it. I don't think it would be
    a problem for the GC not to be able to browse through
    objects still under constructions because such objects are
    never part of a dead cycle anyway, and probably not part of
    a cycle at all until later mutated.
11. brettcannon commented on Sep 6, 2003

    

    brettcannon

    on Sep 6, 2003

    Member

    More actions

    Logged In: YES
    user_id=357491

    The wording Armin proposes in his patch makes sense to me.
12. arigo commented on Oct 28, 2003

    

    arigomannequin

    on Oct 28, 2003

    MannequinAuthor

    More actions

    Logged In: YES
    user_id=4771

    Checked in:

    Doc/lib/libgc.tex (rev: 1.15)
13. ezio-melotti

    transferred this issue fromon Apr 9, 2022