Skip to content

Add a description about formatting decimal.Decimal with the e/E symbol by __format__. #142019

Description

@1nftf
Test output
# Compare different precision on Decimal.
format(Decimal('0'), ".0E")      '0E+0'
format(Decimal('0'), ".1E")      '0.0E+1'
format(Decimal('0'), ".2E")      '0.00E+2'
format(Decimal('0'), ".3E")      '0.000E+3'
format(Decimal('0'), ".30E")     '0.000000000000000000000000000000E+30'
format(Decimal('-0.0'), ".0E")   '-0E-1'
format(Decimal('-0.0'), ".1E")   '-0.0E+0'
format(Decimal('-0.0'), ".2E")   '-0.00E+1'
format(Decimal('-0.0'), ".3E")   '-0.000E+2'
format(Decimal('-0.0'), ".30E")  '-0.000000000000000000000000000000E+29'

# Compare different 0 values.
format(0, ".3E")                '0.000E+00'
format(0.0, ".3E")              '0.000E+00'
format(-0.0, ".3E")             '-0.000E+00'
format(0j, ".3E")               '0.000E+00+0.000E+00j'
format(Decimal('0'), ".3E")     '0.000E+3'
format(Decimal('-0.0'), ".3E")  '-0.000E+2'

# Compare different format methods.
format(Decimal('0'), ".30E")                          '0.000000000000000000000000000000E+30'
format(1234512345123451234512345, ".30E")             '1.234512345123451205320704000000E+24'
format(Decimal('1234512345123451234512345'), ".30E")  '1.234512345123451234512345000000E+24'
"%.30E" % (Decimal('0'),)                             '0.000000000000000000000000000000E+00'  # exponent is 0, because the value is converted to float first
"%.30E" % (1234512345123451234512345,)                '1.234512345123451205320704000000E+24'
"%.30E" % (Decimal('1234512345123451234512345'),)     '1.234512345123451205320704000000E+24'

NOTE: The internal exponent of the Decimal, rather than its sign, is causing the difference.
It seams that precision will affect exponent.

  • When formatting Decimal('0'), the exponent is equal to precision.
  • When formatting Decimal('-0.0'), the exponent is equal to precision-1.

In addition to the above behavior, the exponent is not padded to
two digits, which also makes it inconsistent with the built-in types.

Although the results are numerically correct, and the document
(https://docs.python.org/3/library/string.html#format-specification-mini-language)
does not limit the exponent when the coefficient is 0.
However, this can be confusing for users.

Maybe the document needs to add a description.note, or change the result of Decimal type
to be consistent with built-in types.

Test script
from decimal import Decimal

templates = (
    'format({v}, ".{p}E")',
    # 'f"{{{v}:.{p}E}}"',
    # '"{{:.{p}E}}".format({v})',
    # '"%.{p}E" % ({v},)',

    # 'format({v}, ".{p}e")'
    # 'f"{{{v}:.{p}e}}"',
    # '"{{:.{p}e}}".format({v})',
    # '"%.{p}e" % ({v},)',
)

values = (
    # 0,
    # 0.0,
    # -0.0,
    # complex(0),
    Decimal('0'),
    Decimal('-0.0'),

    # 0.001,
    # Decimal('0.001'),

    # 1,
    # 1.0,
    # Decimal('1'),

    # 10,
    # 10.0,
    # Decimal('10'),

    # 100,
    # 100.0,
    # Decimal('100'),

    # 1234512345123451234512345,
    # 1234512345123451234512345.0,
    # Decimal('1234512345123451234512345'),
)

precision = (
    0,
    1,
    2,
    3,
    # 4,
    # 5,
    # 6,
    # 7,
    # 8,
    # 9,
    # 10,
    30,
    # 100,
    # 1000,
    # 10000,
)

results = [
    ((expr := t.format(v=repr(v), p=p)), eval(expr))
    for t in templates
        for v in values
            for p in precision
]

max_len_expr = max(len(expr) for expr, val in results)
for expr, val in results:
    print(f"{expr}{' ' * ((max_len_expr)-len(expr))}  {repr(val)}")

Linked PRs

Activity

  1. added
    docsDocumentation in the Doc dir
    on Nov 27, 2025
  2. changed the title [-]When format decimal.Decimal('0') to string, the exponent of result is not 0.[/-] [+]When format decimal.Decimal('0') to scientific notation with given precision, the exponent of result is not 0.[/+] on Nov 27, 2025
  3. mdickinson commented on Nov 28, 2025

    @mdickinson
    Member

    Decimal instances differ from floats in that they have significant trailing zeros. For example, Decimal("10"), Decimal("10.0") and "Decimal("1e1") are distinct (although equal under ==) and the difference affects the output of some operations. So the base exponent of a Decimal instance is an extra piece of information beyond just the numerical value.

    To the extent that it's possible, formatting tries to preserve this extra information. That's the reason for the behaviour you're seeing here.

    >>> x = Decimal('0')
    >>> s = format(x, '.2e')
    >>> s
    '0.00e+2'
    >>> y = Decimal(s)
    >>> x.as_tuple() == y.as_tuple()
    True

    Above we've reconstructed y from its formatted representation, and the reconstructed y has exactly the same internal representation as x had. If s were "0.00e+0" instead, we would have lost the exponent information.

    In addition to the above behavior, the exponent is not padded to
    two digits, which also makes it inconsistent with the built-in types.

    This is a separate issue: I'd recommend opening a separate tracker issue if you want to pursue this. But this behaviour is mandated by the specification that the decimal module implements, so changing it would be a tough sell. From the spec (emphasis mine):

    this comprises the letter ‘E’ followed immediately by the adjusted exponent converted to a character form. The latter is in base ten, using the characters 0 through 9 with no leading zeros, always prefixed by a sign character

  4. mdickinson commented on Nov 28, 2025

    @mdickinson
    Member

    Note that the spec itself does not prescribe how .e-style formatting should work: it describes two operations: to-scientific-string, which roughly corresponds to g-style formatting (but without precision control), and to-engineering-string, which is like e-style formatting (again with no precision control) but with the displayed exponent constrained to be a multiple of 3.

    So there's some reading-between-the-lines necessary to implement full-fledged float-style formatting from Decimal.

    But the spirit of preserving the exponent information is present in the to-engineering-string description. E.g., in the to-engineering-string specification:

    if the number is a zero, the zero will have a decimal point and one or two trailing zeros added, if necessary, so that the original exponent of the zero would be recovered by the to-number conversion.

  5. serhiy-storchaka commented on Nov 28, 2025

    @serhiy-storchaka
    Member

    Decimal represents not only an approximate value, but also its precision. Decimal('0') is not the same as Decimal('0.000'). The former represents a value between Decimal('-0.5') and Decimal('0.5'), the latter -- between Decimal('-0.0005') and Decimal('0.0005'). Formatting tries to preserve this property.

    >>> format(Decimal('0'), ".2E")
    '0.00E+2'
    >>> Decimal('0.00E+2')
    Decimal('0')
    >>> format(Decimal('0.000'), ".2E")
    '0.00E-1'
    >>> Decimal('0.00E-1')
    Decimal('0.000')

    So this is not a bug, but a feature.

  6. added
    pendingThe issue will be closed if no feedback is provided
    on Nov 28, 2025
  7. 1nftf commented on Nov 28, 2025

    @1nftf
    ContributorAuthor

    This issue is more about formatting values with the e/E symbol (with a given precision),
    rather than converting the value to a scientific or engineering string.

    When converting/serializing, you might want to keep as much information as possible.

    When formatting, you usually intend to display them in a uniform format.

    # These
    format(Decimal('0.0'), ".3E")    '0.000E+2'
    f"{Decimal('0.0'):.3E}"          '0.000E+2'
    "{:.3E}".format(Decimal('0.0'))  '0.000E+2'
    "%.3E" % (Decimal('0.0'),)       '0.000E+00'  # the Decimal is implicitly converted to a float
    
    # Instead of these
    Decimal('0.0').to_eng_string())             '0.0'
    getcontext().to_eng_string(Decimal('0.0'))  '0.0'
    getcontext().to_sci_string(Decimal('0.0'))  '0.0'
    

    For users who are not familiar with the Decimal class (like me), it
    might be expected that Decimal behaves similarly to float.
    "0.000E+2" rather than "0.000E+00" is indeed a bit strange, and it
    took me a little time to find the reason.

    Documents compare

    These two documents only define how to convert decimal floating-point
    numbers to strings, not how to format them.

    The formatting behavior is defined in these documents.

    The cpp document requires the exponent to be 0 when the value is 0,
    and requires the exponent to be padded to two digits. These is also
    the behavior of python float.

    However the python document omitted above requirements.
    I'm not sure if this is intentionally relaxed for Decimal, or the
    Decimal.__format__ implementer only followed the simplified python
    document.

    Proposal

    My opinion is that we can add a more detailed description of the
    different behavior.

    But if possible, we can make Decimal's format behavior more similar to
    float, and then update the documentation to describe the printf-like
    behavior.


    P.S. I never said this is a bug. The tag I gave this issue from the
    beginning was #docs, not #type-bug .

  8. changed the title [-]When format decimal.Decimal('0') to scientific notation with given precision, the exponent of result is not 0.[/-] [+]"Add a note about formatting decimal.Decimal with the e/E symbol."[/+] on Nov 28, 2025
  9. changed the title [-]"Add a note about formatting decimal.Decimal with the e/E symbol."[/-] [+]Add a note about formatting decimal.Decimal with the e/E symbol.[/+] on Nov 28, 2025
  10. skirpichev commented on Nov 29, 2025

    @skirpichev
    Member

    I think this might be closed.

    Though, maybe new issue should be opened, about minimum number of digits in the exponent for 'e'/'g' formats. I don't think that the decimal spec forbids here using the printf-style formatting also for Decimal's. The Decimal constructor also accepts padding zeros in the exponent. The only question is: does it worth a backward compatibility break?

  11. 1nftf commented on Nov 30, 2025

    @1nftf
    ContributorAuthor

    But a non-zero exponent will make the padding less useful.
    And the '%' formatting will yield a different result due to implicit conversion to float.

    >>> format(1 / Decimal('inf'), ".2e")
    '0.00e-1000024'
    >>> "%.2e" % (1 / Decimal('inf'))
    '0.00e+00'
  12. 1nftf commented on Nov 30, 2025

    @1nftf
    ContributorAuthor

    If current Decimal format behavior is considered a feature and will
    not change in future. A description like these can be added.

    When the value is equal to zero, the exponent is always zero for float,
    and internal exponent plus number of digits after decimal point for
    Decimal.

  13. 1nftf commented on Nov 30, 2025

    @1nftf
    ContributorAuthor
    Comment posted on the wrong issue

    I discovered the formatting difference while investigating #142019

    I found this document first:
    https://docs.python.org/3/library/string.html#format-specification-mini-language.

    When I tried to find another document, I searched for '% format'
    and '% formatting' but found nothing related to
    https://docs.python.org/3/library/stdtypes.html#printf-style-string-formatting.

    I misunderstood that there is only one document about formatting,
    and that the basic functionalities of the format-specification mini-language are also applicable to % formatting,
    except for some additional functions.

    Perhaps a note could be added at the beginning of
    https://docs.python.org/3/library/string.html#format-specification-mini-language
    stating that this is not suitable for % formatting, along with a link back to it."

  14. 5 remaining items

  15. removed
    pendingThe issue will be closed if no feedback is provided
    on Nov 30, 2025
  16. 1nftf commented on Nov 30, 2025

    @1nftf
    ContributorAuthor

    I think I can complete this part and make it unambiguous.

    I've already spent a lot of time on it.

    As current observed behavior
    (only checked on windows, most of x64 releases, some of x86 releases):

    • If float value equal to 0, the exponent is also ​0​
    • If Decimal value equal to 0, the exponent might not be ​0​
    • float will padding exponent to 2digits
    • Decimal always have not padding exponent
  17. 1nftf commented on Nov 30, 2025

    @1nftf
    ContributorAuthor

    After figuring all this out, I will create a PR and highlight the key routines.

    I have little to no experience with real-world C projects, thus need an experienced contributor to help me review it.

  18. skirpichev commented on Dec 1, 2025

    @skirpichev
    Member

    I will create a PR and highlight the key routines.

    Please don't. Could you just modify your pr to include a simple note, as suggested above: "For float the exponent always contains at least two digits."?

  19. 1nftf commented on Dec 1, 2025

    @1nftf
    ContributorAuthor

    This is what most users would expect:

    For float, the exponent always contains at least two digits.

    I would like to add description like this:

    For a given Decimal object obj, if obj == 0, the exponent will be
    obj.as_tuple()[2] + p instead of always being 0.

    @skirpichev
    If you’re on board with this, I’ll go ahead and change the PR.

  20. changed the title [-]Add a note about formatting decimal.Decimal with the e/E symbol.[/-] [+]Add a description about formatting decimal.Decimal with the e/E symbol by __format__.[/+] on Dec 1, 2025
  21. 1nftf commented on Dec 1, 2025

    @1nftf
    ContributorAuthor

    See this comment for aggregated information.

    Sorry for my verbose comment and bad layout and some mistakes,
    made this issue a little off topic.

    I have already made a lots of fold and revision, to make more
    easy to read.

  22. skirpichev commented on Dec 5, 2025

    @skirpichev
    Member

    For a given Decimal object obj, if obj == 0, the exponent will be obj.as_tuple()[2] + p instead of always being 0.

    I'm not sure if this is a good idea. There is nothing special in handling zero for Decimals, as it was explained above. Maybe: "For float the exponent always contains at least two digits and it's zero if the value is zero."

  23. removed their assignment
    on Dec 5, 2025
  24. added a commit that references this issue on Dec 16, 2025
  25. added 2 commits that reference this issue on Dec 16, 2025
  26. added 2 commits that reference this issue on Dec 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    docsDocumentation in the Doc dir

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions