Skip to content

@kvcache-ai kvcache.ai

Change the repository type filter

All

    Repositories list

    10 repositories

    • ktransformers

      Public
      A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
      Python
      •
      Apache License 2.0
      •1.2k•16k•393•3•Updated Jan 6, 2026Jan 6, 2026

    • Mooncake

      Public
      Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
      inferencerdmadisaggregationllmvllmsglangkvcache
      C++
      •
      Apache License 2.0
      •496•4.5k•228•74•Updated Jan 6, 2026Jan 6, 2026

    • sglang

      Public
      SGLang is a fast serving framework for large language models and vision language models.
      Python
      •
      Apache License 2.0
      •4k•5•0•1•Updated Jan 6, 2026Jan 6, 2026

    • DeepEP_fault_tolerance

      Public
      DeepEP: an efficient expert-parallel communication library that supports fault tolerance
      Cuda
      •
      MIT License
      •1.1k•3•0•0•Updated Jan 5, 2026Jan 5, 2026

    • sglang_awq

      Public
      SGLang is a fast serving framework for large language models and vision language models.
      Python
      •
      Apache License 2.0
      •4k•1•0•0•Updated Dec 17, 2025Dec 17, 2025

    • gpustack

      Public
      GPU cluster manager for optimized AI model deployment
      Python
      •
      Apache License 2.0
      •441•0•0•0•Updated Dec 7, 2025Dec 7, 2025

    • TrEnv-X

      Public
      Go
      •
      Apache License 2.0
      •2•72•0•0•Updated Sep 15, 2025Sep 15, 2025

    • sglang-npu

      Public
      SGLang is a fast serving framework for large language models and vision language models.
      Python
      •
      Apache License 2.0
      •4k•0•0•0•Updated Aug 12, 2025Aug 12, 2025

    • custom_flashinfer

      Public
      FlashInfer: Kernel Library for LLM Serving
      Cuda
      •
      Apache License 2.0
      •624•5•0•0•Updated Jul 24, 2025Jul 24, 2025

    • vllm

      Public
      A high-throughput and memory-efficient inference and serving engine for LLMs
      Python
      •
      Apache License 2.0
      •12k•14•0•0•Updated Mar 27, 2025Mar 27, 2025