Repository navigation
Closed
Labels
difficulty: hardLeaderboard: 10 points when the PR closing this is mergedLeaderboard: 10 points when the PR closing this is mergedenhancementNew feature or requestNew feature or requesthackathonOpen for the Secfoo hackathonOpen for the Secfoo hackathonhelp wantedExtra attention is neededExtra attention is needed
Description
A sensor that runs inside a Python app and writes #53 events while the app serves real traffic.
Usage:
secfoo instrument --watchlist watchlist.json -o events.ndjson -- python app.py
# or, when you can't change the start command:
SECFOO_WATCHLIST=watchlist.json python -m secfoo.sensor app.pyApproach:
- Python ≥ 3.12:
sys.monitoring(PEP 669) with its own tool ID. UsePY_STARTevents on the watched functions' code objects only, so unwatched code costs nothing. Returnsys.monitoring.DISABLEafter recording the first stack sample, and keep counting through a cheap path. - Python 3.10/3.11: a fallback using
sys.setprofilefiltered to the watched code objects. It is slower, so document the overhead. - Resolve watched symbols after their modules are imported (an import hook, or resolve lazily on first import).
- Threads:
sys.monitoringcovers all threads, but the 3.10/3.11 fallback needsthreading.setprofiletoo. Forked workers (gunicorn, uvicorn): each process writes its own session file. Document how this works with gunicorn--preload. - Flush to disk every N seconds and at exit. Never block the request path on I/O.
- Coverage: count executed app functions (
PY_STARTon app code objects, disabled after the first hit) forapp_functions_executed.
Must not: read frame.f_locals, arguments or return values; raise into app code; or open any network connection other than SECFOO_SENSOR_ADDR.
Done when: tests cover a sample app calling and not calling a watched function, a crashing sensor that must not take down the app, and a multi-process run. An overhead benchmark (requests/sec with and without the sensor) goes in the PR description.
Added for the live sensor (#51 phase 2):
- Output: follow the Probe ↔ sensor protocol and conformance kit (one contract for every runtime) #73 protocol. When
SECFOO_SENSOR_ADDRis set, send aggregated lines to the sensor (Standalone runtime sensor (secfoo sensor) as sidecar or host daemon #67) over the address it names (Unix socket, loopback TCP, named pipe, or direct HTTPS for serverless), falling back toSECFOO_EVENTS_OUT. Never block on the transport: use a bounded in-memory queue, drop on overflow and count drops. - Watchlist reload: re-read
SECFOO_WATCHLISTwhen its mtime changes (check at most every 30 s) and enable monitoring for newly watched code objects. - Inventory: emit
loadlines (Runtime package inventory: loaded versions, drift and live advisories #68). - Route and auth tagging: read the request context from Attribute runtime hits to live traffic (route, auth state) #69 when it's present.
- Start without code changes: support starting from
sitecustomize(Deploy the runtime sensor anywhere: containers, Kubernetes, VMs, Windows, PaaS, serverless #71). - This probe is the reference implementation for the Probe ↔ sensor protocol and conformance kit (one contract for every runtime) #73 conformance kit, and must pass it.
Metadata
Metadata
Assignees
Labels
difficulty: hardLeaderboard: 10 points when the PR closing this is mergedLeaderboard: 10 points when the PR closing this is mergedenhancementNew feature or requestNew feature or requesthackathonOpen for the Secfoo hackathonOpen for the Secfoo hackathonhelp wantedExtra attention is neededExtra attention is needed
Activity
karthickmk2001 commentedon Nov 12, 2026
I'd like to take this, together with the #73 protocol it's the reference for. PR to follow.
Merge pull request #99 from secfoo-com/feat/55-python-probe