Enhance inspect.getdoc to follow inheritance chains

@ncoghlan

BPO	15582
Nosy	@ncoghlan, @merwok, @PCManticore, @ericsnowcurrently, @berkerpeksag, @vadmium, @serhiy-storchaka, @1st1
Files	inspect_getdoc.patch issue15582.patch: Add documentation. issue15582_1.patch: Ups, remove unintented change. inspect_getdoc_inherited.patch getdoc-news.patch getdoc-news.v2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2015-07-30.15:12:21.015>
created_at = <Date 2012-08-08.02:52:13.350>
labels = ['type-feature', 'library']
title = 'Enhance inspect.getdoc to follow inheritance chains'
updated_at = <Date 2015-07-30.15:12:20.988>
user = 'https://github.com/ncoghlan'

bugs.python.org fields:

activity = <Date 2015-07-30.15:12:20.988>
actor = 'berker.peksag'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2015-07-30.15:12:21.015>
closer = 'berker.peksag'
components = ['Library (Lib)']
creation = <Date 2012-08-08.02:52:13.350>
creator = 'ncoghlan'
dependencies = []
files = ['33118', '34445', '34733', '38414', '39379', '40062']
hgrepos = []
issue_num = 15582
keywords = ['patch']
message_count = 26.0
messages = ['167654', '167909', '206045', '215593', '232322', '232323', '234171', '237691', '237694', '237716', '237760', '240027', '240041', '240057', '240058', '240349', '241489', '241520', '241608', '241726', '241753', '242005', '243248', '247645', '247680', '247681']
nosy_count = 9.0
nosy_names = ['ncoghlan', 'eric.araujo', 'Claudiu.Popa', 'python-dev', 'eric.snow', 'berker.peksag', 'martin.panter', 'serhiy.storchaka', 'yselivanov']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue15582'
versions = ['Python 3.5']

Currently, if you override a method from a base class, the docstring is not inherited, even if it remains accurate.

This issue proposes an enhancement to inspect.getdoc() that allows the docstring to be retrieved from the inheritance hierarchy in the case where it is not overridden in the subclass by providing an explicit docstring.

Specifically, in the case where obj.__doc__ is None, and either the first parameter is a bound method, or a class object is passed in as the second parameter, inspect.getdoc will search the MRO based on obj.__name__ until it finds an attribute with a non-None __doc__ value.

(In Python 2, this could have been automatic for both bound and unbound methods. Unfortunately, there are no unbound methods in Python 3, so the second parameter is needed to handle the unbound method case)

I have just the thing for this, but haven't had a chance to put a patch together. I might be able to get to it in the next week if Dave "bitey" Beazley isn't too much of a distraction. :)

Hello!

Here's a patch. Currently it lacks doc updates, but if the solution is okay, then I could provide them.

Yury, Nick, how is my latest patch?

@Claudiu

@Claudiu, you should also add this test and make sure that it passes:

   class Parent:
      __doc__ = 'some documentation'

   class Child(Parent):
      __doc__ = None

assert getdoc(Child) is None

In other words -- we use __doc__ defined in parent classes only when there was no __doc__ in children's __dict__s.

Alright, I'll update my patch later this week.

Yury, regarding your last message, is it actually possible to have a subclass which doesn't have a __doc__ attribute in its __dict__, except using slots? __doc__ seems to be set to None every time if it's not specified, so I don't know how could I detect the case where the client sets '__doc__ = None' himself.

The following example might be more explanatory:

>>> class A:
...    __doc__ = "a"
...
>>> inspect.getdoc(A)
'a'
>>> inspect.getdoc(A())
'a'
>>> class B(A):
...   __doc__ = None
...
>>> vars(B)
mappingproxy({'__doc__': None, '__module__': '__main__'})
>>> B.__dict__
mappingproxy({'__doc__': None, '__module__': '__main__'})
>>> class C(A): pass
...
>>> vars(C)
mappingproxy({'__doc__': None, '__module__': '__main__'})
>>>

Nevertheless, my patch ignores this case, since it operates only on methods. When trying to do inspect.getdoc(Child, parent=Parent), it will try to look for an attribute 'Child' in the mro of Parent and thus it will return None, since this doesn't exist (this can actually be a problem, if that attribute actually exist).

The patch adds support only for unbound methods and requires additional parameter for this. It is not what should be done in this issue at all.

I'm interested in this issue so I'll write a patch. It should be easier to write it myself than describe it.

The patch adds support only for unbound methods and requires additional parameter for this.

Just curios, which part works only for unbound methods?

It is not what should be done in this issue at all.
It should be easier to write it myself than describe it.

Well, some hints for the right way to do this would have been appreciated.

Here is a patch that supports classes, bound and unbound methods, class methods, properties, method and class descriptors.

Serhiy's patch looks good to me.

I'd completely missed that __qualname__ could be used to avoid needing a second argument even when handling objects other than bound methods. That's very cool, and I can see why you figured it was easier to just write the patch than explain what you had in mind.

New changeset 157b7055bb9d by Serhiy Storchaka in branch 'default':
Issue bpo-15582: inspect.getdoc() now follows inheritance chains.
https://hg.python.org/cpython/rev/157b7055bb9d

I filed issue bpo-23900 to consider the question of the default docstring for Enum subclasses.

Sometimes the doc string for the overridden method does not make much sense in the context of the subclass. Just wondering if this was considered; it seems like a fairly serious downside to this new feature. E.g. in a package I am reviewing, there is a class that inherits HTMLParser and converts HTML to PDF. There is no doc string, so previously there was just the signature in the “pydoc” output. Now the output looks like:

| __init__(self, pdf, image_map=None)
| Initialize and reset this instance.
|
| If convert_charrefs is True (the default), all character references
| are automatically converted to the corresponding Unicode characters.

The second paragraph mentions parameters and settings of the internal base class, which doesn’t make much sense for the subclass.

Sounds like good incentive to add good docstrings. :)

I were aware that this can propagate some not well appropriate docstrings from abstract base classes, but Martin expose worse problem: inheriting a docstring by the method with changed signature.

Perhaps we should check if a signature of overriding method is compatible with a signature of base method. But this is hard to implement, and impossible in some cases until all builtins will converted to Argument Clinic.

I know it will take some time to add doc strings to places where they are missing, but I would not roll-back the patch for that. File new issues and get the over-riding methods properly documented.

My example was taken from a package that tends to “properly document” everything in separate documentation files, rather than in the source code. I don’t really think Python should dictate if and how one document their code based on what base classes they use, and pydoc is still useful with code that has no doc strings.

But if this is to be the way of the future, perhaps a warning in the “What’s New” page might be a good idea. The inspect.getdoc() documentation should probably also have a “Changed in version 3.5” warning.

I guess it would be less automatic, but maybe another option might have been to add an @inherit_docstring decorator, or some explicit way to say a particular API (re)implements an (abstract) API documented elsewhere.

Yes, this was a deliberate change to "flip the default" as to which subclasses get bad docstring behaviour.

In the status quo, if you provide a subclass method which does basically the same thing as the parent class, you lose the docstring unless you duplicate it, meaning you have to choose between bad docstrings and a maintainability problem due to content duplication.

With this change, you only need a custom docstring in the subclass if you've changed the method behaviour enough that the parent class docstring no longer makes any sense.

If you just want to suppress the docstring entirely, then you'll need to specify an empty docstring.

This is potentially worth a note in the "Porting to Python 3.5" section of the What's New document, but it's an intended consequence of the change.

getdoc-news.patch suggests some wording to add to What’s New, and also adds a “Changed in version 3.5” note to inspect.getdoc().

BTW I also noticed that the class doc strings are not inherited from object.__doc__, although method doc strings _are_ inherited from object(), such as object.__init__.__doc__. The current documentation suggests that the class doc string “The most base type” should also be inherited.

$ cat module.py
class SomeClass:
    '''CLASS DOCSTRING'''
    def __init__(self):
        '''METHOD DOCSTRING'''
$ ./python -m pydoc module.SomeClass  # Doc strings intact
[. . .]
module.SomeClass = class SomeClass(builtins.object)
 |  CLASS DOCSTRING
 |  
 |  Methods defined here:
 |  
 |  __init__(self)
 |      METHOD DOCSTRING
 |  [. . .]
$ ./python -OOm pydoc module.SomeClass  # Method inherited, class stripped
[. . .]
module.SomeClass = class SomeClass(builtins.object)
 |  Methods defined here:
 |  
 |  __init__(self)
 |      Initialize self.  See help(type(self)) for accurate signature.
 |  [. . .]

I also wonder how well this feature would work when someone tries to override a base method by using a mix-in type class.

Merged the current What’s New page in getdoc-news.v2.patch. Is there any interest in applying this?

New changeset 4476b578b8fd by Berker Peksag in branch '3.5':
Issue bpo-15582: Add a whatsnew entry for inspect.getdoc() changes in 3.5.
https://hg.python.org/cpython/rev/4476b578b8fd

New changeset e0f4a5f09bfa by Berker Peksag in branch 'default':
Issue bpo-15582: Add a whatsnew entry for inspect.getdoc() changes in 3.5.
https://hg.python.org/cpython/rev/e0f4a5f09bfa

Applied, thanks!

Navigation Menu

Repository navigation

Enhance inspect.getdoc to follow inheritance chains #59787

3 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Enhance inspect.getdoc to follow inheritance chains #59787

Description

Activity

ncoghlan commented on Aug 8, 2012

ericsnowcurrently commented on Aug 10, 2012

PCManticore commented on Dec 13, 2013

PCManticore commented on Apr 5, 2014

1st1 commented on Dec 8, 2014

PCManticore commented on Dec 8, 2014

PCManticore commented on Jan 17, 2015

serhiy-storchaka commented on Mar 9, 2015

PCManticore commented on Mar 9, 2015

serhiy-storchaka commented on Mar 9, 2015

ncoghlan commented on Mar 10, 2015

python-dev commented on Apr 3, 2015

3 remaining items

ncoghlan commented on Apr 9, 2015

vadmium commented on Apr 19, 2015

ethanfurman commented on Apr 19, 2015

serhiy-storchaka commented on Apr 20, 2015

ethanfurman commented on Apr 21, 2015

vadmium commented on Apr 21, 2015

ncoghlan commented on Apr 25, 2015

vadmium commented on May 15, 2015

vadmium commented on Jul 30, 2015

python-dev commented on Jul 30, 2015

berkerpeksag commented on Jul 30, 2015

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions