Skip to content
This repository was archived by the owner on Aug 13, 2026. It is now read-only.
This repository was archived by the owner on Aug 13, 2026. It is now read-only.

Crash with google.protobuf.json_format.ParseError: Value '2024-09-23' has unexpected type <class 'datetime.date'> #943

Closed
Closed
Crash with google.protobuf.json_format.ParseError: Value '2024-09-23' has unexpected type <class 'datetime.date'>#943
Assignees
gkevinzheng
Labels
api: loggingIssues related to the googleapis/python-logging API.

Description

@daaain
daaain
opened on Sep 23, 2024

Environment details

  • OS type and version: python:3.11.9-bookworm base image
  • Python version: 3.11.9
  • pip version: pip 24.0 from /usr/local/lib/python3.11/site-packages/pip (python 3.11)
  • google-cloud-logging version: 3.11.2

Steps to reproduce

Trying to send down a dict that has a DateTime property blows the library up with a google.protobuf.json_format.ParseError

I guess I could work around it manually and convert it to a string myself, but the bigger issue is that I wouldn't expect a logging library to crash my application.

Ideally the library should handle anything thrown at it with a best effort string conversion (even if it's just a useless class instance identity or anything), but at the very least have a way to log its own error rather than throwing an unhandled exception? Or are end users expected to wrap it in try / except and handle certain errors? Or is there a way to give the library a custom JSON parser?

Code example

from datetime import datetime
from google.cloud import logging

logging_client = logging.Client()
logging_client.setup_logging()

my_dict = {"my_date": datetime.now()}

logging_client.logger("default").log_struct({"message": "message", "my_dict": my_dict}, severity="INFO")

I thought maybe using the logging std lib might make a difference but that also crashes:

import logging
from datetime import datetime
from google.cloud import logging as gcloud_logging

logging_client = gcloud_logging.Client()
logging_client.setup_logging()

data_dict = {"my_date": datetime.now()}
logging.info("message field", extra={"json_fields": data_dict})

>>> Failed to submit 1 logs.
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 600, in _ConvertFieldValuePair
    self.ConvertMessage(value, sub_message, '{0}.{1}'.format(path, name))
  File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 485, in ConvertMessage
    methodcaller(_WKTJSONMETHODS[full_name][1], value, message, path)(self)
  File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 704, in _ConvertStructMessage
    self._ConvertValueMessage(value[key], message.fields[key],
  File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 682, in _ConvertValueMessage
    raise ParseError('Value {0} has unexpected type {1} at {2}'.format(
google.protobuf.json_format.ParseError: Value 2024-09-23 17:55:34.185216 has unexpected type <class 'datetime.datetime'> at LogEntry.jsonPayload.my_date

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/google/cloud/logging_v2/handlers/transports/background_thread.py", line 115, in _safely_commit_batch
    batch.commit()
  File "/usr/local/lib/python3.11/site-packages/google/cloud/logging_v2/logger.py", line 468, in commit
    client.logging_api.write_entries(
  File "/usr/local/lib/python3.11/site-packages/google/cloud/logging_v2/_gapic.py", line 154, in write_entries
    log_entry_pbs = [_log_entry_mapping_to_pb(entry) for entry in entries]
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/google/cloud/logging_v2/_gapic.py", line 154, in <listcomp>
    log_entry_pbs = [_log_entry_mapping_to_pb(entry) for entry in entries]
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/google/cloud/logging_v2/_gapic.py", line 557, in _log_entry_mapping_to_pb
    ParseDict(mapping, entry_pb)
  File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 446, in ParseDict
    parser.ConvertMessage(js_dict, message, '')
  File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 487, in ConvertMessage
    self._ConvertFieldValuePair(value, message, path)
  File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 616, in _ConvertFieldValuePair
    raise ParseError(str(e)) from e
google.protobuf.json_format.ParseError: Value 2024-09-23 17:55:34.185216 has unexpected type <class 'datetime.datetime'> at LogEntry.jsonPayload.my_date

Stack trace

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 600, in _ConvertFieldValuePair
    self.ConvertMessage(value, sub_message, '{0}.{1}'.format(path, name))
  File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 485, in ConvertMessage
    methodcaller(_WKTJSONMETHODS[full_name][1], value, message, path)(self)
  File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 704, in _ConvertStructMessage
    self._ConvertValueMessage(value[key], message.fields[key],
  File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 670, in _ConvertValueMessage
    self._ConvertStructMessage(value, message.struct_value, path)
  File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 704, in _ConvertStructMessage
    self._ConvertValueMessage(value[key], message.fields[key],
  File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 682, in _ConvertValueMessage
    raise ParseError('Value {0} has unexpected type {1} at {2}'.format(
google.protobuf.json_format.ParseError: Value 2024-09-23 17:24:50.706366 has unexpected type <class 'datetime.datetime'> at LogEntry.jsonPayload.my_dict.my_date

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.11/site-packages/google/cloud/logging_v2/logger.py", line 220, in log_struct
    self._do_log(client, StructEntry, info, **kw)
  File "/usr/local/lib/python3.11/site-packages/google/cloud/logging_v2/logger.py", line 169, in _do_log
    client.logging_api.write_entries(entries, partial_success=True)
  File "/usr/local/lib/python3.11/site-packages/google/cloud/logging_v2/_gapic.py", line 154, in write_entries
    log_entry_pbs = [_log_entry_mapping_to_pb(entry) for entry in entries]
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/google/cloud/logging_v2/_gapic.py", line 154, in <listcomp>
    log_entry_pbs = [_log_entry_mapping_to_pb(entry) for entry in entries]
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/google/cloud/logging_v2/_gapic.py", line 557, in _log_entry_mapping_to_pb
    ParseDict(mapping, entry_pb)
  File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 446, in ParseDict
    parser.ConvertMessage(js_dict, message, '')
  File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 487, in ConvertMessage
    self._ConvertFieldValuePair(value, message, path)
  File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 616, in _ConvertFieldValuePair
    raise ParseError(str(e)) from e
google.protobuf.json_format.ParseError: Value 2024-09-23 17:24:50.706366 has unexpected type <class 'datetime.datetime'> at LogEntry.jsonPayload.my_dict.my_date```

Activity

  1. product-auto-label
    added
    api: loggingIssues related to the googleapis/python-logging API.
    on Sep 23, 2024
  2. blunderbuss-gcf
    assigned
    bhshkh
    on Sep 23, 2024
  3. daaain
    changed the title [-]Crash with google.protobuf.json_format.ParseError: Value '2024-11-25' has unexpected type <class 'datetime.date'>[/-] [+]Crash with google.protobuf.json_format.ParseError: Value '2024-09-23' has unexpected type <class 'datetime.date'>[/+] on Sep 23, 2024
  4. gkevinzheng
    assigned
    gkevinzheng
    and unassigned
    bhshkh
    on Sep 24, 2024

  5. gkevinzheng commented on Jan 27, 2025

    @gkevinzheng
    gkevinzheng
    on Jan 27, 2025
    Contributor

    @daaain:

    We decided to, in the case of ParseErrors like this, rethrow a ValueError with that error message, so you will have to catch this on your own. This is because logging internally might cause a breaking change. We also made more explicit what the format of the dictionary that gets passed into log_struct should me in #958 as well.

    This is because log_struct tries to convert the Python dictionary that gets passed in into a Protobuf Struct, which doesn't accept Python datetime objects. To work around this, you're probably going to have to use strftime on each datetime object.

    I could see something about using custom JSON parsing similar to json.dumps.

  6. gkevinzheng
    closed this as completedon Jan 27, 2025

  7. daaain commented on Jan 27, 2025

    @daaain
    daaain
    on Jan 27, 2025
    Author

    Had a look at the PR, makes sense!

    One thing I was wondering, would it be possible to add type hints so that MyPy or other static analysis libraries can catch if an unserialisable value is being passed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

api: loggingIssues related to the googleapis/python-logging API.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions