Read-only Telegram data proxy with FastAPI, Telethon and an optional official Bot API provider. Its only write is joining a public channel or supergroup on request (muted and archived) so it can be monitored. The independently installed tg CLI calls the deployed HTTP API and provides JSON-first access for agents. Both packages release together.
Future contributors should start with AGENTS.md and the maintainer context.
- FastAPI REST API with OpenAPI at
/openapi.json,/schema/openapi.json, and/schema/openapi.yaml - Long-lived Telethon session stored on a mounted volume for Docker and Ubuntu deployments
- Read-only v1 API for chats, contacts, self info, message retrieval, context, search, media manifests, streaming downloads, and ZIP bundles
- Telegram safety defaults: low
flood_sleep_threshold, serialized history-heavy reads, cached entity resolution, and one-session-owner deployment guidance - Local-only e2e harness for a personal Telegram account, plus unit/integration tests with 80% coverage enforcement
- Encrypted Business/Secretary update cache, verified owner identity, provider health and response provenance
- Schema-driven
tgcommands, a bundled agent skill, shell completion and enforced API/CLI parity - Channel monitoring: batched newest-message ids in one Telegram request and a server-sent event stream of new messages
Requires Python 3.12+ on macOS or Linux. Download the installer from the latest GitHub release, then run it:
python3 install-tg.py
tg login --url https://your-api.example
tg auth-status --remote
tg chats list
tg messages search --query "meeting"
tg operations listThe installer verifies the CLI wheel checksum, installs into an isolated environment under ~/.local/share/tg, and links ~/.local/bin/tg. Put ~/.local/bin on PATH. It installs the bundled skill globally and shell completions; unrelated existing files are preserved. Use --version X.Y.Z to install an exact release. tg login prompts for the API bearer token without echo; this is separate from Telegram account authentication. For automation, use TG_API_URL and TG_API_TOKEN or --token-stdin. Credentials are stored with mode 0600 and bound to one API URL. See CLI documentation.
TELEGRAM_BOT_MODE=off preserves Telethon-only operation. shadow starts Bot polling and encryption but serves all public reads through Telethon. primary prefers Bot metadata and exact observed private messages, with capability-aware fallback. History, contacts, dialogs, search, context, albums and bundles always use Telethon; a cache miss is never proof that a message does not exist.
Before enabling collection, the owner must confirm that the disclosed proxy/cache use complies with the current Telegram Bot Developer Terms, including Business data restrictions. Keep mode off until that activation gate is met. Implementation and tests do not enable live collection.
- Create a bot with BotFather, enable Secretary Mode and configure its privacy policy to disclose this private proxy and retention period.
- Set
TELEGRAM_BOT_TOKEN,TELEGRAM_BOT_OWNER_USER_IDto the same account as Telethon, andTELEGRAM_BOT_STORE_KEYto a separately stored random base64-encoded 32-byte key. Never store the key with the SQLite database or commit either secret. - Start in
shadow, connect the owner account to the bot and select eligible private chats. If no connection is known yet, disconnect/reconnect after polling starts so Telegram sends its connection update. Existing webhooks must be removed explicitly by the operator; startup never deletes them. - Check
/healthz: both providers healthy, poller running and connection verified. Compare read responses, then switch toprimary. Roll back withoff.
Generate the cache key locally and transfer it directly to your secret manager:
python3 -c 'import base64,secrets; print(base64.urlsafe_b64encode(secrets.token_bytes(32)).decode())'Messages expire after TELEGRAM_BOT_RETENTION_DAYS (default 30, range 1–365). Connection edits/disable and missed-update windows clear cached messages. Every record is authenticated/encrypted with AES-GCM and indexed with opaque HMAC keys. A failed poll disables cached reads until catch-up completes; 401 and 429 never fall back to evade authentication or cooldowns. Only one process may own the bot poller and Telethon session. Keep the cache on the persistent service volume. Key rotation means disabling Bot mode and deleting only the disposable Bot cache before starting with a new key; preserve the Telethon session.
JSON responses include meta.provider and meta.fallback_used; downloads include X-Telegram-Provider and X-Telegram-Fallback. Health retains its HTTP 200 response and reports degraded providers in its body. Enable API_AUTH_ENABLED=true with a strong token on any non-local deployment.
- Create Telegram API credentials at my.telegram.org.
- Create a local env file:
cp .env.example .env- Authenticate once and persist the session volume:
docker compose --profile auth run --rm telegram-auth- Start the API:
docker compose up -d telegram-proxyThe API will be available at http://localhost:4040 by default. Set PORT to change it; the app also still accepts APP_PORT for compatibility. Compose publishes the port on 127.0.0.1 only, because Docker's port rules bypass host firewalls such as ufw; expose it to other machines through a reverse proxy or tailscale serve.
make install
make auth
make runThe auth CLI stores the Telethon SQLite session under TELEGRAM_SESSION_DIR/TELEGRAM_SESSION_NAME.session.
For local development, .env.example uses ./data/telegram so make auth writes to a project-local directory instead of a root-owned path.
Only one long-lived service instance should own a Telegram session for a given account. Running the same session from multiple hosts or containers risks AUTH_KEY_DUPLICATED, which invalidates the session.
Telegram publishes no request limits for user accounts, so the service applies conservative local budgets to keep the personal account away from bursts and repeated username lookups:
| Variable | Default | Effect |
|---|---|---|
TELEGRAM_MAX_CONCURRENT_REQUESTS |
2 |
Telegram operations in flight at once. |
TELEGRAM_MIN_REQUEST_INTERVAL |
0.35 |
Minimum seconds between operation starts. |
TELEGRAM_DOWNLOAD_CONCURRENCY |
2 |
Concurrent media downloads, streams and ZIP bundles. |
TELEGRAM_RESOLVE_LIMIT / TELEGRAM_RESOLVE_WINDOW |
20 / 3600 |
New usernames, phones or invite links resolved through Telegram per rolling window. |
TELEGRAM_RESOLVE_NEGATIVE_TTL |
3600 |
Seconds a name Telegram could not resolve is answered locally as not found (0 disables). |
TELEGRAM_JOIN_LIMIT / TELEGRAM_JOIN_WINDOW |
20 / 3600 |
New channel joins per rolling window. |
TELEGRAM_LATEST_NONMEMBER_TTL |
600 |
Seconds a peer that broke a batched chats/latest request is skipped (0 disables). |
TELEGRAM_RECEIVE_UPDATES |
true |
Receive passive updates for /updates/stream; false makes the stream return 503. |
TELEGRAM_UPDATES_QUEUE_LIMIT / TELEGRAM_UPDATES_PING_INTERVAL |
1000 / 15 |
Per-stream backlog before the stream is closed; seconds between : ping comments. |
Usernames already in the session database never reach Telegram. A username that is not cached is resolved only by endpoints that explicitly allow it (/resolve, from_user, common chats); other endpoints return 404 and ask for the numeric ID. An exhausted local budget returns 429 with Retry-After and type local-rate-limit without contacting Telegram. Telegram flood waits of every kind return 429 with Retry-After; a frozen account returns 403 with type account-frozen.
GET /healthzGET /meGET /contactsGET /resolve?value=@usernameGET /chatsGET /chats/latest?chat_ids=-100…,-100…POST /chats/{chat_id}/joinGET /chats/{chat_id}GET /chats/{chat_id}/photoGET /users/{user_id}/common-chatsGET /chats/{chat_id}/messagesGET /chats/{chat_id}/messages/{message_id}GET /chats/{chat_id}/messages/{message_id}/contextGET /messages/searchGET /chats/{chat_id}/messages/{message_id}/mediaGET /chats/{chat_id}/messages/{message_id}/media/{media_id}GET /chats/{chat_id}/messages/{message_id}/media/bundleGET /updates/stream?chat_ids=-100…,-100…(text/event-stream)
Chat ids are the marked ids /resolve returns (-100… for channels and supergroups).
POST /chats/{chat_id}/joinjoins a public channel or public supergroup, then mutes and archives it, returning{chat_id, already_member, muted, archived}. It is idempotent: an existing membership is reported with its current mute/archive state and never changed. Mute/archive failures only show up asfalse. Users and basic groups are 400; private, invite-only, approval-gated or banned chats are 403not-joinable; flood waits and the local join budget are 429.GET /chats/latest?chat_ids=…(1–100 distinct ids, request order kept) returns{chat_id, member, top_message_id}per id using one Telegram request for all member chats. Chats the account has not joined or never saw reportmember: falsewithout extra requests.GET /updates/stream?chat_ids=…(1–500 ids) sendsevent: readyonce subscribed,event: messagewith{chat_id, message_id, grouped_id, date}for each new message in a requested chat, and a: pingcomment every 15 s. Edits, deletions, service messages and unrequested chats are never sent. A stream that falls too far behind is closed; reconnect and re-sync withchats/latest. Delivery is best-effort: Telegram may throttle passive updates for some channels, so keep pollingchats/latestas a fallback.
messagesandmessages/searchacceptfrom_user,since(inclusive),until(exclusive) andbefore_id; naive times are UTC.queryis optional on search whenfrom_useris set, andfrom_useron search requireschat_id.- Filtered queries use cursor pagination: pass
meta.next_before_idasbefore_iduntil it is null;pagemust stay 1. Unfiltered queries keep page pagination and also return the cursor. - Groups filter senders on Telegram's side. Telegram ignores sender filters in private chats, so those are read newest-first and filtered locally, stopping at
since. include_raw=falseomitstelegram_rawfrom message summaries and their forward headers.users/{user_id}/common-chatslists groups and channels both accounts belong to; it excludes the private chat and chats either side has left.
GET /.../mediareturns a manifest of downloadable media items.include_album=truecollects sibling messages that share the same Telegramgrouped_id.GET /.../media/{media_id}streams a single file with binary headers.GET /.../media/bundlereturns a ZIP archive containing every file plusmanifest.json.GET /chats/{chat_id}/photoreturns the chat's small profile photo (normallyimage/jpeg) with the same binary headers, or 404 when the chat has no photo. It always uses Telethon.
Install dependencies:
make installThis creates a local .venv automatically and installs all dependencies there, so it works with Homebrew-managed Python environments that block system-wide pip install.
Run tests:
make testRun the complete API/CLI contract, lint, format, strict typing and test gates:
make checkAfter changing an endpoint or schema, regenerate the bundled contract with .venv/bin/python scripts/export_openapi.py. Commit its diff together with command coverage, tests and skill/documentation changes. CI rejects stale schemas and uncovered API operations. Python 3.12 and 3.14 are tested.
requirements.lock supplies the same dependency constraints to local installs, Docker, CI and releases. To update dependencies intentionally, run uv pip compile pyproject.toml cli/pyproject.toml --extra dev --universal --output-file requirements.lock, reinstall, regenerate the schema if necessary and run the complete gate.
Run local real-account e2e tests:
export TELEGRAM_E2E_ENV_FILE=/absolute/path/to/.env
export E2E_CHAT_ID=123
export E2E_TEXT_MESSAGE_ID=456
export E2E_MEDIA_MESSAGE_ID=789
make test-e2eFor the Bot path, additionally set TELEGRAM_BOT_E2E=1 and E2E_BOT_MESSAGE_ID to an already observed message in the selected private chat, using an explicitly configured primary environment. These tests never send messages. Stop the deployed instance before running local e2e against its session/token; never create a second session owner or poller.
Install the git hook:
make pre-commit-installThe pre-commit hook runs scripts/check.py, including the configured 80% API coverage threshold. Real-account tests remain opt-in.
A successful CI run for a push to main triggers the Release workflow. It selects the next minor version from published/tagged versions (x.(y+1).0), builds the API and CLI from the same commit and publishes wheels, source distributions, checksums and the installer to GitHub Releases. The workflow also supports Run workflow → version for an explicit higher X.Y.Z version on main. Validation runs for automatic and manual releases. Version stamps are applied only to the build checkout, avoiding release commit loops. Re-running an already published commit is a no-op; interrupted draft releases can resume.
- Copy
.envto the server and keep it out of version control. - Run the auth profile once to create the persisted session volume.
- Keep the named Docker volume or bind-mount
/data/telegramso restarts do not require re-authentication. - Upgrade with
git pull --ff-only && GIT_REVISION=$(git rev-parse HEAD) docker compose up -d --build, then confirm the running container'sorg.opencontainers.image.revisionlabel matches that commit:docker compose upkeeps an existing container when no image change reaches it. - Builds use the default bridge network and need working DNS there. If
pip installfails withTemporary failure in name resolutionwhile running containers resolve fine, the host's resolver probably lists IPv6 servers first: glibc uses only three nameservers and build containers have no IPv6. Set IPv4 servers in"dns"in/etc/docker/daemon.jsonand restart Docker.