Visualizes and debugs multi-step LLM agent workflows with interactive graphs and local inspection
β¨ Features β’ β‘ Quick start β’ π§° Usage β’ π§© Architecture β’ π Roadmap β’ π€ Contributing
AI agents chain dozens of LLM calls, tool invocations, memory reads, and nested logic, yet failures remain opaque black boxes. Cloud observability platforms are expensive and framework-specific, while log-based tools drown you in noise. FlowTrace is a lightweight, local-first dashboard that auto-instruments any Python agent code and renders clean interactive SVG graphsβso you can instantly see what happened, why it failed, and how to fix it.
- Zero-config tracing with a single
@tracedecorator or context managerβno framework lock-in - Official LangChain and LlamaIndex handlers for zero-code instrumentation of any chain, agent, LLM, query engine or tool
- Interactive SVG node-link diagrams that visualise nesting, timing, step kinds, and data flow on a strict grid
- Clickable inspector panels showing raw prompts, responses, token counts, tool arguments, latency, and errors
- Local SQLite persistence with history browsing, trace replay (mocked tools), HTTP ingestion for non-Python agents, and summary statistics
- Export traces to OpenTelemetry (OTLP) and LangSmith compatible formats for interoperability with other observability platforms
|
π Zero-config tracing Decorator, context manager, and official LangChain + LlamaIndex handlers |
π Interactive SVG graphs Strict hierarchical grid showing flow, nesting and timing |
π Prompt inspector Click any node for raw prompts, responses, tokens and metadata |
π₯ Latency flamegraph & token analytics Interactive stacked timeline + per-LLM token bar charts with click-to-inspect |
|
π οΈ Tool call browser Every invocation with arguments, output and caller location |
πΎ Local persistence SQLite store with history, content search across prompts & tool outputs, filtering, side-by-side comparison |
Re-execute saved traces with mocked tools |
π‘ HTTP ingestion POST endpoint for non-Python or remote agents |
|
π΄ Live streaming WebSocket endpoint β traces appear live as they run |
π Summary statistics Token usage, success rate, slowest steps and failure patterns |
π Side-by-side comparison Select any two traces to view aligned SVG graphs, stats, and computed differences |
|
|
π Standards export OpenTelemetry OTLP JSON and LangSmith run format (CLI, API, Python) |
|||
Requires Python 3.9+. No external services or database servers needed.
pip install flowtraceFor LangChain support:
pip install "flowtrace[langchain]"For LlamaIndex support:
pip install "flowtrace[llamaindex]"Start the Swiss-minimal dashboard:
flowtrace serveOpen http://127.0.0.1:8080. The server also registers the /api/traces endpoints automatically.
from flowtrace import trace, current_trace
@trace("ResearchAgent")
def research(query: str):
tc = current_trace()
tc.record_llm_call(
prompt=f"Summarize research on: {query}",
response="Agents need observability...",
tokens=124,
latency=0.38
)
tc.record_tool_call(
name="wikipedia",
args={"query": query},
result={"summary": "Key concepts..."}
)
return "final answer"
from flowtrace import TraceContext
with TraceContext(name="DemoWorkflow", live=True):
research("LLM observability")Run flowtrace serve in another terminal and visit the UI. Your trace appears instantly (and builds live via WebSocket) with an interactive graph.
With LangChain (zero-code):
# pip install "flowtrace[langchain]" langchain-core langchain-openai
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
from flowtrace import FlowTraceHandler
handler = FlowTraceHandler(name="ResearchAgent", live=True)
llm = ChatOpenAI(model="gpt-4o-mini", callbacks=[handler])
@tool
def wikipedia(query: str):
"""Look up information on Wikipedia."""
return "Agent observability is essential..."
# Pass the handler to any chain/agent/LLM
# agent = create_react_agent(llm=llm, tools=[wikipedia], ...)
# agent.invoke(...) # trace saved automaticallyWith LlamaIndex (zero-code):
# pip install "flowtrace[llamaindex]" llama-index-core llama-index
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llama_index.core.callbacks import CallbackManager
from flowtrace import LlamaIndexHandler
handler = LlamaIndexHandler(name="QueryEngine", live=True)
Settings.callback_manager = CallbackManager([handler])
documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
response = query_engine.query("Explain FlowTrace observability")
print(response)from flowtrace import TraceContext
with TraceContext(name="CustomerSupportAgent") as tc:
tc.record_step("Load user profile", "setup")
tc.record_step("Log event") # kind defaults to "step"
with tc.step("Generate response", kind="phase", metadata={"model": "gpt-4"}):
tc.record_llm_call("You are a helpful agent...", "Here is your refund link", 87)
tc.record_tool_call("get_order", {"order_id": 42}, {"status": "shipped"})Enable real-time WebSocket updates so the dashboard graph builds node-by-node while your agent runs (works with both manual TraceContext and the LangChain/LlamaIndex handlers):
from flowtrace import TraceContext, current_trace
with TraceContext(name="LiveResearchAgent", live=True) as tc:
tc.record_llm_call(...)
tc.record_tool_call(...)
# UI updates instantly via /api/ws/traces/{id}Set FLOWTRACE_SERVER_URL=http://127.0.0.1:9000 if you use a non-default port.
from flowtrace import FlowTraceHandler
from langchain_core.callbacks import BaseCallbackHandler
handler = FlowTraceHandler(name="MyAgent", live=True)
# Pass to any LangChain component
llm = ChatOpenAI(callbacks=[handler])
# or agent = initialize_agent(..., callbacks=[handler])
# or chain = prompt | llm with config={"callbacks": [handler]})The handler automatically:
- Creates a
TraceContexton first event - Builds a nested step tree using LangChain's
run_id/parent_run_id - Records LLM prompts/responses/tokens, tool inputs/outputs, chain steps
- Finalizes timing and persists the trace to SQLite when the root run completes
from flowtrace import LlamaIndexHandler
from llama_index.core.callbacks import CallbackManager
from llama_index.core import Settings
handler = LlamaIndexHandler(name="MyIndexWorkflow", live=True)
Settings.callback_manager = CallbackManager([handler])
# Any LlamaIndex query, chat, agent, embedding or retrieval now emits traces
# index.as_query_engine().query(...)
# agent.chat(...)The handler automatically:
- Creates a
TraceContexton the first event - Maps LlamaIndex
CBEventType(LLM, QUERY, RETRIEVE, EMBEDDING, TOOL, AGENT, ...) into steps - Parses payload for prompts, responses, tokens, queries and errors
- Builds a nested tree using event IDs and persists on root completion
After selecting a trace, click the Analytics tab (or call the /api/traces/{id}/analytics endpoint directly). You will see:
- Latency flamegraph β interactive SVG timeline with horizontal bars sized by duration, vertically stacked by nesting depth, colored by step kind. Hover and click any bar to jump to the inspector.
- Per-step token analytics β SVG bar chart of all LLM calls sorted by token usage, with numeric labels and a summary panel.
- A sortable table of every step with latency and token counts (click rows to inspect).
All visualizations are built with vanilla SVG + JS and stay consistent with the Swiss-minimal design.
In the dashboard, use the checkboxes next to traces in the history list to select exactly two traces, then click Compare in the header. The main view switches to a split layout with:
- Two interactive SVG graphs placed side-by-side (strict hierarchical grid layout preserved for each trace)
- Summary statistics for both traces
- A differences panel showing duration delta, token usage delta, and step count delta
- A shared inspector: click any node in either graph to see its prompt/response/tool data, with visual indication of which trace it belongs to
Exit comparison mode with the Single View button to return to the standard inspector+graph layout. This makes it trivial to spot regressions, performance changes, or differences in agent behaviour across runs.
Any saved trace can be exported to OpenTelemetry OTLP JSON (for collectors, Jaeger, Tempo, etc.) or LangSmith's run format.
Via CLI:
flowtrace export 8f3c9a2b-... --format otel --output trace.json
flowtrace export 8f3c9a2b-... --format langsmith --output runs.jsonIn Python:
from flowtrace import export_to_opentelemetry, export_to_langsmith
from flowtrace.storage import SQLiteStore
store = SQLiteStore()
trace = store.get_trace("8f3c9a2b-...")
otel_data = export_to_opentelemetry(trace)
langsmith_runs = export_to_langsmith(trace)
import json
with open("otel.json", "w") as f:
json.dump(otel_data, f, indent=2)
with open("langsmith.json", "w") as f:
json.dump(langsmith_runs, f, indent=2)The OTEL export produces a full resourceSpans payload with proper trace/span IDs, timestamps (derived from wall-clock + perf_counter offsets), attributes, status and semantic conventions (e.g. gen_ai.completion.token_count). The LangSmith export produces a flat list of runs with run_type, inputs/outputs, parent_run_id, trace_id, extra, etc.
The /api/traces/{id}/export?format=otel endpoint is also available.
flowtrace list --limit 10
flowtrace view 8f3c9a2b-...
flowtrace serve --port 9000 --host 0.0.0.0import requests
requests.post(
"http://127.0.0.1:8080/api/traces",
json={...} # Trace model shape from models.py
)Advanced: Custom step kinds and error recording
with tc.step("MemoryRead", "memory", {"key": "user:123"}):
...
# record_step can omit the kind (defaults to "step")
tc.record_step("Quick note")
# Errors are automatically captured by the context manager
with tc.step("RiskyAPI"):
raise ValueError("Rate limit exceeded")Advanced: Replaying a saved trace with mocks
Load a trace via the UI or flowtrace view, then replay it while overriding tool calls using the same record_tool_call hooks with deterministic results.
| Option | Default | Description |
|---|---|---|
port |
8080 |
Port the dashboard listens on (flowtrace serve --port 9000 or run_server(port=9000)) |
host |
"127.0.0.1" |
Bind address for the server (--host 0.0.0.0 for external access) |
db_path |
"flowtrace.db" |
SQLite file path passed to SQLiteStore(db_path=...) |
limit |
20 (CLI) / 50 (API) |
Number of traces returned by list_traces(limit=...) or flowtrace list --limit 50 |
live |
false |
Enable real-time WebSocket streaming (TraceContext(..., live=True) or FlowTraceHandler(..., live=True) or LlamaIndexHandler(..., live=True); respects FLOWTRACE_SERVER_URL env var) |
flowchart TD
AgentCode["Agent / examples/agent_example.py"] --> Tracer["tracer.py<br/>TraceContext + @trace + Step tree<br/>(live=True streams via HTTP)"]
LangChain["LangChain (LLM/Agent/Chain)"] --> Handler["langchain.py<br/>FlowTraceHandler (official callback)"]
LlamaIndex["LlamaIndex (QueryEngine/Agent/LLM)"] --> Handler2["llamaindex.py<br/>LlamaIndexHandler (official callback)"]
Handler --> Tracer
Handler2 --> Tracer
Tracer --> Models["models.py<br/>Pydantic Trace + Step (with .from_tracer_context)"]
Models --> Storage["storage.py<br/>SQLiteStore (save_trace, get_trace, list_traces)"]
Storage <--> DB["flowtrace.db"]
Storage --> API["api.py<br/>FastAPI router (/api/traces, /graph, POST ingest, POST /live, WS /ws/traces/{id}, GET /compare/{id1}/{id2}, GET /analytics/{id})"]
API --> Server["server.py<br/>create_app() + Jinja templates + static mount"]
Server --> UI["templates/index.html + static/script.js<br/>SVG renderer + inspector + live WS client + side-by-side compare + flamegraph + token chart"]
CLI["cli.py<br/>serve, list, view, clear"] --> Server
External["Non-Python agents"] --> API
current_trace["current_trace() ContextVar"] -.-> Tracer
TraceContext builds a nested Step tree using perf_counter timing and contextvars for async/thread safety. The tree is converted to Pydantic models, persisted as JSON in SQLite (with denormalised summary fields), and served to a single-page Swiss-minimal frontend that renders an interactive SVG graph on a strict hierarchical grid. All public APIs (trace, TraceContext, run_server, FlowTraceHandler, LlamaIndexHandler, CLI commands) are defined in the shown source files.
.
βββ pyproject.toml # Metadata, dependencies, console script entrypoint
βββ src/flowtrace/
β βββ __init__.py # Public API: trace(), TraceContext, current_trace(), run_server(), FlowTraceHandler, LlamaIndexHandler
β βββ langchain.py # Official LangChain BaseCallbackHandler (zero-code)
β βββ llamaindex.py # Official LlamaIndex BaseCallbackHandler (zero-code)
β βββ tracer.py # Core TraceContext, Step dataclass, @trace decorator, record_llm_call/record_tool_call
β βββ models.py # Pydantic Trace, Step (with from_tracer_context), TraceSummary
β βββ storage.py # SQLiteStore with create_tables, save_trace, get_trace, list_traces, clear_all
β βββ api.py # FastAPI router: /api/traces (GET/POST), /{id}, /{id}/graph
β βββ cli.py # Typer CLI: serve(--port, --host), list(--limit), view, clear
β βββ server.py # create_app() that mounts static files, Jinja templates and API router
β βββ ...
βββ templates/index.html # Single-page Swiss-minimal UI (12-column grid, sidebar, SVG canvas, inspector)
βββ static/
β βββ script.js # Vanilla JS: loadTraces(), renderGraph() (strict grid SVG), onNodeClick(), inspector, renderFlamegraph(), renderTokenChart()
β βββ styles.css # Strict Swiss design: #e30613 accent, hairline rules, system-ui, no shadows
βββ tests/test_tracer.py # pytest suite for context, decorator, nesting, timing, errors, uniqueness, LangChain + LlamaIndex handlers
βββ examples/agent_example.py # Runnable mock research agent demonstrating realistic usage + LangChain handler
pip install -e ".[test,langchain,llamaindex]"
pytestThe test suite (tests/test_tracer.py) covers TraceContext nested step construction, @trace decorator behaviour (both top-level and nested), LLM/tool recording, timing accuracy, error capture, current_trace() contextvar semantics, UUID uniqueness, both official handlers (FlowTraceHandler and LlamaIndexHandler with mocked callbacks and store), and the new analytics methods on Trace.
- v0.1.0 β first release: Zero-config tracing, Interactive graphs, Prompt inspector, Tool call browser
- Add official LangChain callback handler for zero-code instrumentation
- Implement WebSocket endpoint for live trace streaming during runs
- Build side-by-side trace comparison view
- Add search across prompts and tool outputs in trace history
- Create first-party LlamaIndex integration adapter
- Export traces to OpenTelemetry / LangSmith compatible format
- Add latency flamegraph and per-step token analytics dashboard
- Support manual annotations and comments on any step
- Implement trace replay with swapped model parameters or mocked outputs
- Add plugin system for custom step visualizers and badges
- Provide official Docker image with one-command deployment
- Support non-Python agents via OpenTelemetry auto-instrumentation
π€ This roadmap is worked through continuously β ideas and PRs welcome!
Contributions are welcome and appreciated! Please read CONTRIBUTING.md and adhere to the CODE_OF_CONDUCT.md. Good first issues are tagged and make excellent PR starting points. Feel free to open an issue or PR against the Swiss-minimal dashboard, the SVG layout algorithm, or the tracer internals.
MIT Β© FlowTrace Contributors. See LICENSE for details.