Skip to content

Python live sensor (sys.monitoring) #55

Closed

Description

@vsn411

A sensor that runs inside a Python app and writes #53 events while the app serves real traffic.

Usage:

secfoo instrument --watchlist watchlist.json -o events.ndjson -- python app.py
# or, when you can't change the start command:
SECFOO_WATCHLIST=watchlist.json python -m secfoo.sensor app.py

Approach:

  • Python ≥ 3.12: sys.monitoring (PEP 669) with its own tool ID. Use PY_START events on the watched functions' code objects only, so unwatched code costs nothing. Return sys.monitoring.DISABLE after recording the first stack sample, and keep counting through a cheap path.
  • Python 3.10/3.11: a fallback using sys.setprofile filtered to the watched code objects. It is slower, so document the overhead.
  • Resolve watched symbols after their modules are imported (an import hook, or resolve lazily on first import).
  • Threads: sys.monitoring covers all threads, but the 3.10/3.11 fallback needs threading.setprofile too. Forked workers (gunicorn, uvicorn): each process writes its own session file. Document how this works with gunicorn --preload.
  • Flush to disk every N seconds and at exit. Never block the request path on I/O.
  • Coverage: count executed app functions (PY_START on app code objects, disabled after the first hit) for app_functions_executed.

Must not: read frame.f_locals, arguments or return values; raise into app code; or open any network connection other than SECFOO_SENSOR_ADDR.

Done when: tests cover a sample app calling and not calling a watched function, a crashing sensor that must not take down the app, and a multi-process run. An overhead benchmark (requests/sec with and without the sensor) goes in the PR description.

Added for the live sensor (#51 phase 2):

Activity

  1. added
    enhancementNew feature or request
    help wantedExtra attention is needed
    hackathonOpen for the Secfoo hackathon
    difficulty: hardLeaderboard: 10 points when the PR closing this is merged
    on Nov 8, 2026
  2. karthickmk2001 commented on Nov 12, 2026

    @karthickmk2001
    Collaborator

    I'd like to take this, together with the #73 protocol it's the reference for. PR to follow.

  3. added a commit that references this issue on Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    difficulty: hardLeaderboard: 10 points when the PR closing this is mergedenhancementNew feature or requesthackathonOpen for the Secfoo hackathonhelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions