Skip to content
vanshdev0101Public

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

2 Commits

Folders and files

Repository files navigation

Orbit — Distributed Job Scheduler

A distributed job scheduling platform: Postgres schema → Express/TypeScript REST API → Node.js worker service (polling, atomic claiming, retries, DLQ) → Next.js dashboard ("Orbit"). Built as a college internship assignment. Evaluation prioritizes backend engineering, DB design, and concurrency/reliability over frontend polish — the dashboard exists to demonstrate the backend, not the other way around.

Architecture

Dashboard (Next.js, polling) ──REST──▶ API Service (Express/TypeScript) │ ┌───────────────────┼───────────────────┐ ▼ ▼ ▼ PostgreSQL Worker Service(s) Cron Dispatcher (source of truth) poll→claim→run→report (leader-elected) ▲ │ └── Stale-Claim Reaper (reclaims dead workers' jobs)

Full rationale for this shape — including why Postgres-as-queue instead of a broker, why polling over WebSockets, and every other trade-off — lives in docs/architecture.md and docs/design-decisions.md.

Tech stack

  • Backend: Express + TypeScript, pg (raw SQL, no ORM), zod validation, jsonwebtoken + bcryptjs.
  • Worker: plain Node + axios + cron-parser, no framework.
  • Frontend: Next.js 14 (Pages Router), plain CSS + <style jsx>, Chart.js for the throughput chart. No Tailwind, no component library.
  • Database: PostgreSQL 15+.
  • Testing: Jest + ts-jest.
  • Package management: npm workspaces (backend, worker, frontend).

Repository structure

job-scheduler/ ├── db/schema.sql # full Postgres schema — run this first ├── docs/ # architecture, ER diagram, API reference, design decisions ├── tests/ # root-level Jest tests (NOT inside backend/ or frontend/) ├── backend/ # Express API ├── worker/ # polling worker + cron dispatcher + stale-claim reaper └── frontend/ # Next.js dashboard

See docs/architecture.md for the full annotated tree and file-by-file breakdown.

Setup

1. Database

createdb orbit
psql -d orbit -f db/schema.sql

Requires PostgreSQL 15+ (uses gen_random_uuid() via the pgcrypto extension, created automatically by the schema script).

2. Install dependencies

From the repository root (not inside backend/ — npm workspaces hoist shared dependencies, and tests/ can't resolve backend packages if you install from within backend/ alone):

npm install

3. Environment variables

Copy each .env.example to .env and fill in real values:

cp backend/.env.example backend/.env
cp worker/.env.example worker/.env
cp frontend/.env.example frontend/.env

Key variables to check:

  • backend/.env → DATABASE_URL, JWT_SECRET, FRONTEND_URL (must exactly match the port the frontend actually runs on — a mismatch produces a generic "Failed to fetch" in the browser with no useful server error, indistinguishable from the backend being down).
  • worker/.env → API_BASE_URL, API_KEY (issued via POST /auth/api-keys after registering a user).
  • frontend/.env → NEXT_PUBLIC_API_BASE.

4. Run each service (separate terminals)

# Backend
cd backend && npm run dev

# Worker
cd worker && npm run dev

# Frontend
cd frontend && npm run dev

5. Run tests

From the repo root:

npm test

tests/claim.integration.test.ts requires a TEST_DATABASE_URL env var and auto-skips otherwise — this is intentional, not a failure.

Current status

Backend — fully implemented: auth, projects, queues (CRUD + pause/resume/delete + live stats), jobs (create/batch/recurring templates/list/detail/cancel), workers (register/heartbeat/drain/ claim/start/complete/fail), DLQ, metrics. Atomic job claiming via SELECT ... FOR UPDATE SKIP LOCKED. 15/15 unit tests passing.

Frontend — login/register, session-guarded dashboard, Overview tab (live stat cards + 24h throughput chart), Queues tab (live stats, pause/ resume/edit/delete), Job Explorer (create + detail modals, status filters), Workers tab, DLQ tab.

Known gaps (see docs/design-decisions.md and the recommended-next-steps section of the original handoff notes for full context):

  • No GET endpoint to list recurring job templates yet — blocks any recurring-jobs UI (POST /jobs/recurring exists and works, there's just nowhere to see existing templates).
  • No Queue creation modal (POST /queues works via API/curl; only edit/delete have UI so far).
  • No batch job creation UI (POST /jobs/batch works via API).

Full API reference: docs/api.md. Schema and ER diagram: docs/er-diagram.md.

Notes on reliability decisions worth knowing before a demo

  • Queue deletion is guarded: deleting a queue with jobs in it returns 409 with the job count instead of silently cascading — pass ?force=true to proceed. See design-decisions.md §10 for why (the underlying FK is ON DELETE CASCADE, so an unguarded delete would destroy history with no warning).
  • Queue stats (GET /queues, GET /queues/:id/stats) are computed live via a GROUP BY over jobs, not read from a stored counter — see design-decisions.md §9 for the story behind this.
  • A job created with an unregistered job_type will retry until maxAttempts is exhausted, always failing with No handler registered for job_type "...", then move to the dead-letter queue. This is expected — the worker's handler registry (worker/src/handlers.ts) currently only knows example.echo, example.http_fetch, and example.flaky.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages