Allowing filtering/formatting of Stackdriver logs

Hello,

I'm using the instructions here to send Python logging module logs to Stackdriver. Thanks for making this so easy! However, doing this naively seems to make all logs from potentially many jobs/VMs/etc. get merged together in the Stackdriver UI. When I filter to e.g. a specific VM, none of the logs show up (screenshot) -- I have to set the filtering to "Global" to see them (screenshot). Am I doing something wrong?

Also, what about logging things like files, line numbers, etc.? Is this possible? Does Stackdriver respect formatting I apply using the Python logging API? What about automatic links to files in connected code repos etc. -- any info available on that?

Thanks!

@liyanhui1228

@liyanhui1228 Can you point @joshburkart in the right direction?

I figured this out (I think) -- I now make calls to the metadata endpoint to determine info about the current VM, then pass these as labels when making logging handlers. Rough code snippet in case it's useful to someone:

import logging as pylogging

from google.cloud import logging as cloudlogging

...

handler = cloudlogging.handlers.CloudLoggingHandler(
    <stackdriver_client>,
    resource=cloudlogging.resource.Resource(
        # See e.g. https://cloud.google.com/monitoring/api/resources
        # for more info on codified Stackdriver "Resources".
        type='gce_instance',
        labels=<gce_instance labels retrieved from metadata endpoint>,
    ),
    labels=<any extra labels>,
)

logger = pylogging.getLogger('whatever')
logger.addHandler(handler)

(I think this should be added to the documentation or something, though...)

@liyanhui1228

@liyanhui1228 I'm not sure where such an example should go: who manages the snippets / examples used in the cloud.google.com docs?

@dazuma

@dazuma Can you help direct this to the right target for updating the API docs?

@JustinBeckwith

@JustinBeckwith, @theacodes can either of you point me to the person / team responsible for managing the cloud.google.com/logging docs samples?

As of #200, the resource should be auto-detected for each GCP environment. It sounds like that was the ask for this issue, so I'll close this. Feel free to re-open if there are any remaining concerns

Navigation Menu

Repository navigation

Allowing filtering/formatting of Stackdriver logs #17

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Allowing filtering/formatting of Stackdriver logs #17

Description

Activity

tseaver commented on Mar 16, 2018

joshburkart commented on Mar 20, 2018

tseaver commented on Mar 26, 2018

tseaver commented on Oct 18, 2018

tseaver commented on Dec 5, 2018

daniel-sanche commented on Mar 5, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions