Skip to content

dockerflow.logging.JsonLogFormatter is weird about %s style logging #29

Open

Description

@peterbe

I have this config:

    # Logging
    LOGGING_USE_JSON = values.BooleanValue(True)
    LOGGING_DEFAULT_LEVEL = values.Value("INFO")

    @property
    def LOGGING(self):
        return {
            "version": 1,
            "disable_existing_loggers": False,
            "formatters": {
                "json": {
                    "()": "dockerflow.logging.JsonLogFormatter",
                    "logger_name": "buildhub",
                },
                "verbose": {"format": "%(levelname)s %(asctime)s %(name)s %(message)s"},
            },
            "handlers": {
                "console": {
                    "level": self.LOGGING_DEFAULT_LEVEL,
                    "class": "logging.StreamHandler",
                    "formatter": ("json" if self.LOGGING_USE_JSON else "verbose"),
                },
                "sentry": {
                    "level": "ERROR",
                    "class": (
                        "raven.contrib.django.raven_compat.handlers" ".SentryHandler"
                    ),
                },
                "null": {"class": "logging.NullHandler"},
            },
            "root": {"level": "INFO", "handlers": ["sentry", "console"]},

...

When I do:

In [1]: import logging

In [2]: logger = logging.getLogger("myproject")

In [3]: logger.info("Hi!")
{"Timestamp": 1541787363833421056, "Type": "myproject", "Logger": "buildhub", "Hostname": "MacBook-Pro-131.local", "EnvVersion": "2.0", "Severity": 6, "Pid": 68488, "Fields": {"msg": "Hi!"}}

In [4]: logger.info("Hi %s", "peter")
{"Timestamp": 1541787379742094080, "Type": "myproject", "Logger": "buildhub", "Hostname": "MacBook-Pro-131.local", "EnvVersion": "2.0", "Severity": 6, "Pid": 68488, "Fields": {"msg": "Hi peter"}}

In [5]: logger.info("Hi %s", {'name': 'Peter'})
{"Timestamp": 1541787416647823104, "Type": "myproject", "Logger": "buildhub", "Hostname": "MacBook-Pro-131.local", "EnvVersion": "2.0", "Severity": 6, "Pid": 68488, "Fields": {}}

Notice that when the param to logger.debug(template, THIS) isn't a string the final "msg" becomes empty ("Fields": {}).

Activity

  1. peterbe commented on Nov 9, 2018

    @peterbe
    Author

    Contrast that with regular "verbose" logging:

    ▶ DJANGO_LOGGING_USE_JSON=False DJANGO_LOGGING_DEFAULT_LEVEL=DEBUG ./manage.py shell
    Python 3.6.5 (default, Mar 30 2018, 06:41:53)
    Type 'copyright', 'credits' or 'license' for more information
    IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.
    
    In [1]: import logging
    
    In [2]: logger = logging.getLogger("myproject")
    
    In [3]: logger.info("Hi!")
    INFO 2018-11-09 18:19:07,726 myproject Hi!
    
    In [4]: logger.info("Hi %s", "peter")
    INFO 2018-11-09 18:19:13,941 myproject Hi peter
    
    In [5]: logger.info("Hi %s", {'name': 'Peter'})
    INFO 2018-11-09 18:19:20,103 myproject Hi {'name': 'Peter'}
    
  2. peterbe commented on Nov 9, 2018

    @peterbe
    Author

    The "cause" of this is that the record.getMessage() here becomes this string: "Hi {'name': 'Peter'}"
    and because of the line:

    if message and not message.startswith('{') and not message.endswith('}'):

    ...it decides to not include the message.

    I'm trying to parse that code comment two lines above but struggling to understand what it means ... is not already a JSON blob.

  3. peterbe commented on Nov 9, 2018

    @peterbe
    Author

    Because I'm timely desperate for a solution, here's how I solved it:

    In [3]: stuff = {'name': 'Peter'}
    
    In [4]: logger.info(f"Hi {stuff}")
    {"Timestamp": 1541788091760187904, "Type": "myproject", "Logger": "buildhub", "Hostname": "MacBook-Pro-131.local", "EnvVersion": "2.0", "Severity": 6, "Pid": 69542, "Fields": {}}
    
    In [5]: logger.info(f"Hi {stuff} ")
    {"Timestamp": 1541788125543843072, "Type": "myproject", "Logger": "buildhub", "Hostname": "MacBook-Pro-131.local", "EnvVersion": "2.0", "Severity": 6, "Pid": 69542, "Fields": {"msg": "Hi {'name': 'Peter'} "}}
    
    

    By making sure the logging message string has an extra character that is not }, it works.

  4. diox commented on Sep 26, 2019

    @diox
    Contributor

    So if you still care @peterbe #37 will make it easier for you to override this behavior, there is a is_value_jsonlike method you can override on the JsonLogFormatter class.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions