CodeRAG — Chat With Any Codebase

Point it at any GitHub repo and ask questions in plain English. Get accurate, cited answers powered by CodeBERT, FAISS, and Groq Llama 3.

---

What it does

Ask questions about any codebase in plain English and get accurate, cited answers:

• "What does this project do? Give me a high-level overview."
• "How does authentication work in this codebase?"
• "What API endpoints are available and what do they do?"
• "How would I add a new feature following the existing patterns?"

Every answer cites the exact source file and line numbers. A faithfulness score flags when the model has gone off-script.

---

Architecture

GitHub Repo / Local Code / File Upload
         |
  Code-aware Chunker       splits at class/function/section boundaries
         |
  CodeBERT Embedder        microsoft/codebert-base (PyTorch, local)
         |
  +---------------+
  |  FAISS Index  |  <--- Dense retrieval (cosine similarity)
  |  BM25 Index   |  <--- Sparse retrieval (keyword matching)
  +-------+-------+
          |
  Reciprocal Rank Fusion   merges both ranked lists
          |
  Cross-encoder Reranker   ms-marco-MiniLM precision boost
          |
  Groq Llama 3 70B         cited, structured answers
          |
  NLI Faithfulness Check   flags hallucinated sentences
          |
  FastAPI (port 8001)      REST API
  Next.js UI (port 3000)   dark-mode chat interface with source viewer
  MLflow (port 5000)       logs every query, score, and run

---

Features

Feature	Details
Per-repo isolation	Each repo gets its own FAISS index — no cross-contamination
Hybrid retrieval	Dense (CodeBERT) + Sparse (BM25) merged via Reciprocal Rank Fusion
Two-stage reranking	Cross-encoder for high-precision results
Persistent index	Survives restarts — no re-embedding needed
Multi-repo switching	Switch between ingested repos with the command palette (Cmd+K)
Faithfulness scoring	NLI model detects hallucinated answers
All file types	Python, JS, TS, HTML, CSS, Go, Rust, Java, and more
Source viewer	Expandable syntax-highlighted code preview for every citation
MLflow tracking	Every query logged with scores, retrieval stats, and parameters
Fine-tuning script	Train CodeBERT on CodeSearchNet with contrastive loss

---

Quickstart (local)

1. Clone and install

git clone https://github.com/yourusername/coderag
cd coderag

python3.11 -m venv .venv
source .venv/bin/activate

pip install --upgrade pip
pip install torch==2.2.2
pip install "numpy<2"
pip install -r requirements.txt

2. Configure

cp .env.example .env
# Edit .env and add your GROQ_API_KEY (free at https://console.groq.com)

3. Install frontend dependencies

cd frontend && npm install && cd ..

4. Start everything

chmod +x start.sh
./start.sh

Services:

Service	URL
UI	http://localhost:3000
API	http://localhost:8001
API Docs	http://localhost:8001/docs
MLflow	http://localhost:5000

5. Ingest a repo and ask

Click No repository selected in the topbar, paste a GitHub URL, and start asking questions.

---

Deployment

Vercel (frontend) + Railway (backend)

Step 1 — Deploy the backend to Railway

1. Go to railway.app → New Project → Deploy from GitHub repo → select coderag
2. Railway detects railway.json and builds using the root Dockerfile automatically
3. Add these environment variables in the Railway dashboard:

   GROQ_API_KEY=your_key
   GROQ_MODEL=llama-3.3-70b-versatile
   EMBEDDING_MODEL=microsoft/codebert-base
   RERANKER_MODEL=cross-encoder/ms-marco-MiniLM-L-6-v2
   FAISS_INDEX_PATH=data/processed/faiss_index
   CHUNK_SIZE=512
   CHUNK_OVERLAP=64
   TOP_K_RETRIEVE=20
   TOP_K_RERANK=5
   FAITHFULNESS_THRESHOLD=0.5
   PYTHONPATH=/app
   

4. Generate a public domain in Railway settings — copy the URL (e.g. https://coderag-api.up.railway.app)

Step 2 — Deploy the frontend to Vercel

1. Go to vercel.com → New Project → Import the coderag repo
2. Set Root Directory to frontend
3. Add these environment variables:

   NEXT_PUBLIC_API_URL=https://coderag-api.up.railway.app
   NEXT_PUBLIC_MLFLOW_URL=http://localhost:5000
   NEXT_PUBLIC_API_KEY=
   

4. Deploy — copy your Vercel URL (e.g. https://coderag.vercel.app)

Step 3 — Lock down CORS

Back in Railway, add:

ALLOWED_ORIGINS=https://coderag.vercel.app

Redeploy the Railway service. Done.

---

Docker Compose (self-hosted)

cp .env.example .env
# Edit .env — set GROQ_API_KEY at minimum

docker compose up --build

All three services start together. Data is persisted in the named mlflow_data volume and the ./data bind mount.

---

Tech Stack

Component	Technology
Embeddings	microsoft/codebert-base via SentenceTransformers
Vector store	FAISS flat inner-product (cosine, L2-normalised)
Sparse retrieval	BM25 (rank-bm25)
Fusion	Reciprocal Rank Fusion
Reranker	cross-encoder/ms-marco-MiniLM-L-6-v2
Faithfulness	cross-encoder/nli-MiniLM2-L6-H768
LLM	Llama 3.3 70B Versatile via Groq API
Backend	FastAPI + Uvicorn
Frontend	Next.js 16 + TypeScript + Tailwind CSS v4
UI state	Zustand + TanStack Query
Syntax highlighting	Shiki
Experiment tracking	MLflow
Fine-tuning	PyTorch + SentenceTransformers

---

Project Structure

coderag/
├── src/
│   ├── config.py                  # Pydantic settings
│   ├── pipeline.py                # Main orchestrator
│   ├── ingestion/
│   │   ├── chunker.py             # Language-aware chunker
│   │   └── loaders.py             # GitHub, local, text loaders
│   ├── retrieval/
│   │   ├── embedder.py            # CodeBERT + FAISS + BM25 + RRF
│   │   ├── reranker.py            # Cross-encoder reranker
│   │   ├── retriever.py           # Hybrid pipeline
│   │   └── query_rewriter.py      # LLM query expansion
│   ├── generation/
│   │   ├── generator.py           # Groq LLM
│   │   └── faithfulness.py        # NLI hallucination checker
│   ├── evaluation/
│   │   └── evaluator.py           # ROUGE + faithfulness metrics
│   └── api/
│       └── main.py                # FastAPI REST endpoints
├── frontend/                      # Next.js 16 chat UI
│   ├── src/
│   │   ├── app/                   # App router pages + layout
│   │   ├── components/            # Topbar, sidebar, chat, sources
│   │   ├── hooks/                 # useRepos, useIngest, useChat
│   │   ├── lib/                   # api.ts, store.ts, utils.ts
│   │   └── types/                 # Shared TypeScript interfaces
│   ├── Dockerfile                 # Multi-stage production build
│   └── .env.local                 # Local dev defaults
├── scripts/
│   ├── cli.py                     # CLI tool
│   └── finetune_codebert.py       # Fine-tuning script
├── tests/
├── Dockerfile                     # FastAPI backend image
├── docker-compose.yml             # Full stack (API + frontend + MLflow)
├── start.sh                       # One-command local dev startup
├── .env.example                   # All environment variables documented
└── requirements.txt               # Pinned Python dependencies

---

Environment Variables

See .env.example for the full reference. Required variables:

Variable	Required	Description
GROQ_API_KEY	Yes	Groq API key (free at console.groq.com)
GROQ_MODEL	No	Default: llama-3.3-70b-versatile
ALLOWED_ORIGINS	No	CORS origins. Default * (restrict in production)
API_KEY	No	API key auth. Leave blank to disable

---

Fine-tuning CodeBERT

python scripts/finetune_codebert.py \
  --output models/codebert-finetuned \
  --epochs 3 --batch-size 16

# Then in .env:
EMBEDDING_MODEL=models/codebert-finetuned

---

Note on Intel Mac / PyTorch 2.2

Tested on Intel Mac with PyTorch 2.2.2 and Python 3.11. The start.sh script sets all required threading and library environment variables automatically (KMP_DUPLICATE_LIB_OK, OMP_NUM_THREADS, etc.).

---

License

MIT