Admission control for speculative LLM calls on a shared vLLM/SGLang/Tensor-RT LLM server
-
Updated
Nov 13, 2026 - Python
Admission control for speculative LLM calls on a shared vLLM/SGLang/Tensor-RT LLM server
Open-source LLM inference benchmarks — TTFT, TPOT, Throughput, Latency & Cost-per-token for models like Llama, Qwen, Gemma, DeepSeek, Gpt-Oss etc. deployed on different dedicated GPUs.
Implemented the DROID-SLAM visual SLAM pipeline on the NVIDIA Jetson AGX Orin to achieve real-time camera tracking and 3D scene reconstruction on edge hardware. The project focuses on running deep-learning-based SLAM efficiently on an embedded GPU device.
Minimal TensorRT demo on NVIDIA L4 GPUs: Export PyTorch MLP to ONNX, optimize with FP16, and achieve 40%+ latency reduction via batched inference.
To associate your repository with the tensor-rt topic, visit your repo's landing page and select "manage topics."