Skip to content
#

nvlink

Star

Here are 25 public repositories matching this topic...

Language: Python
Filter by language
All 50 Python 25 Cuda 5 C++ 4 C 2 Go 2 Rust 2 TypeScript 2 C# 1 HTML 1 PowerShell 1
Sort: Most stars
Sort options
Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated
local-llm-tuning-guide

Lowaltitudeflight / local-llm-tuning-guide

Star 67

A decision library for running local LLMs on multi-GPU machines without NVLink: parallelism, communication, quantization, speculative decoding, KV cache. Every conclusion carries its evidence grade and reproducible conditions.

benchmarking self-hosted nvidia performance-tuning quantization multi-gpu pcie model-parallelism inference-optimization nvlink kv-cache tensor-parallelism llm llmops vllm local-llm speculative-decoding llm-deployment gpu-serving decision-guide
  • Updated Nov 4, 2026
  • Python

JohnScheuer / comm-cost-modeling

Star 4

Communication cost modeling for tensor parallel LLM inference with TP vs PP vs hybrid comparison, VRAM analysis, pipeline bubble modeling, regime detection, and cost-efficiency. Shows TP dominates on NVLink, PP has 47% bubble at 8 GPUs, and LLaMA-70B needs 8× A100 or 2× H100 for VRAM.

pytorch pcie nvlink communication-cost tensor-parallelism llm-inference gpu-profiling distributed-inference scaling-analysis
  • Updated Jul 15, 2026
  • Python

SiliconLanguage / model-explorer-open-llm

Star 1

A hybrid testbed for evaluating top open-source LLMs (like gpt-oss-20b and Llama 3.3) on local, cloud GPUs, and AWS Inferentia2/Trainium instances, focusing on vLLM optimization, capacity management, kernel bypass, hardware-software co-design, as well as supporting infrastructure such as NCCL, RDMA, NVMeoF.

aws gpu rdma nvme kernel-bypass nccl gpudirect nvlink nvmeof llm vllm trainium vllm-serve inferentia2 software-hardware-co-design aws-ofi-nccl
  • Updated Apr 21, 2026
  • Python

JohnScheuer / pipeline-parallel-kv-bench

Star 1

Analytical benchmark connecting pipeline parallelism and KV cache management: bubble-time overlap for disaggregation, prefix reuse tax of PP stages, and tiering speedup across NVLink, InfiniBand, and PCIe

performance-engineering benchmark simulation inference systems infiniband disaggregation pcie nvlink analytical-model tiering pipeline-parallelism kv-cache llm llm-serving machine-learning-infrastructure prefix-sharing kv-cache-management bubble-time
  • Updated Jul 26, 2026
  • Python

JohnScheuer / tensor-parallel-kv-cache-bench

Star 0

Analytical benchmark connecting tensor parallelism and KV cache management in LLM serving: GQA sharding regimes, all-gather vs recompute tradeoffs, PCIe viability, and operational decision matrix across NVLink, InfiniBand, and PCIe

performance-engineering benchmark simulation inference systems attention infiniband disaggregation pcie nvlink analytical-model gqa kv-cache tensor-parallelism llm llm-serving grouped-query-attention machine-learning-infrastructure prefix-sharing all-gather
  • Updated Jul 26, 2026
  • Python

Add this topic to your repo

To associate your repository with the nvlink topic, visit your repo's landing page and select "manage topics."

Learn more