This repository was archived by the owner on Aug 13, 2026. It is now read-only.
Repository navigation
This repository was archived by the owner on Aug 13, 2026. It is now read-only.
Custom logs from a GCP Function do not contain execution-id #343
Description
I am not sure if this is a bug or a misconfiguration issue but we are unable to get the execution-id in our logs. GCP's own function logs like startup/teardown/timeout contain it however our own logs don't. This makes it impossible to correlate all logs from an instance.
Example (GCP function startup log)
{
insertId: "000000-a51b0cd9-1f4d-4396-b812-4b35c8878b69"
labels: {
execution_id: "j57c7yk340qq"
}
logName: "projects/PROJECT-ID/logs/cloudfunctions.googleapis.com%2Fcloud-functions"
receiveTimestamp: "2021-06-30T17:37:16.202805413Z"
resource: {
labels: {
function_name: "FUNCTION-NAME",
project_id: "PROJECT-ID",
region: "REGION"
}
type: "cloud_function"
}
severity: "DEBUG"
textPayload: "Function execution started"
timestamp: "2021-06-30T17:37:05.341077546Z"
}
We also tried setting a custom label in CloudLoggingHandler with a trace-id however GCP's own logs don't reflect the custom trace-id. This makes me think that GCP is using a separate logger which attaches the execution-id and our logs are isolated from that logger.
Environment details
- OS type and version: N/A
- Python version: 3.7.9
- pip version: 21.1.2
google-cloud-loggingversion: 2.3.1
Steps to reproduce
- Create and attach
CloudLoggingHandlerto a logger in a GCP function - Log an
INFOmessage - Verify the log does not contain
execution-idinsidelabels
Code example
A simplified version of our logging setup and a basic event-trigger function.
import logging
import google.cloud.logging
from google.cloud.logging_v2.handlers.handlers import CloudLoggingHandler
class FunctionLogger(logging.Logger):
def __init__(self, level=logging.NOTSET):
super().__init__("logger_name", level)
client = google.cloud.logging.Client()
self._handler = CloudLoggingHandler(client)
self.propagate = False # prevent logging from root logger
self.addHandler(self._handler)
def handler(event, context=None):
"""function entry point"""
logger = FunctionLogger()
logger.info("Test message")Thanks!!
Activity
daniel-sanche commentedon Jul 1, 2021
The CloudLoggingHandler you are using sends batched logs over the network. This works well in most environments, but it has issues in Cloud Functions, since the code is ephemeral. We recently added StructuredLogHandler as the new default handler for GCF, which prints all logs as structured JSON to stdout for the agent to parse. Using this method should solve your issue.
You can either manually specify StructuredLogHandler in place of CloudLoggingHandler in your existing code, or you can just use
client.setup_logging()and let the library do the recommended set-up based on the environment for you (see docs here)amertkara commentedon Jul 9, 2021
Thanks for the quick response!
StructuredLogHandlermade theexecution-idappear for all logs, however my log message is now printed insidetextPayload. Before, I would pass my message along with a context dictionary andCloudLoggingHandlerwould allow me to log it underjsonPayload. How can I achieve this withStructuredLogHandler?daniel-sanche commentedon Jul 15, 2021
Hmm I didn't think the
CloudLoggingHandlersupported sending arbitraryjsonPayloadsyet. The transport classes are hard-coded to only send a message and a python_logger field (so it is technically a jsonPayload, but you can only control the message field). I'd be interested in seeing the code you used.Full support for
jsonPayloadsis a new feature we are adding in v3.0.0. If you want to try it early, you should be able to install the library from the v3.0.0 branch, or just copy the updatedStructuredLogHandlercode directly from the codebase.