Explore Model Recipes

qwen3-1.7b-llama-cpp

Qwen3 1.7B (Q8_0 GGUF) -- small test model via llama.cpp

qwen3-1.7b-sglang

Qwen3 1.7B -- small test model, solo or cluster (SGLang)

qwen3-1.7b-vllm

Qwen3 1.7B -- small test model, solo or cluster

qwen3-coder-next-fp8-sglang

Qwen3 Coder Next (upstream FP8 quant) -- cluster only

qwen3.5-0.8b-bf16-sglang

Qwen3.5 0.8B (upstream, no quant) SGLang

qwen3.5-122b-a10b-fp8-sglang

Qwen3.5 122B A10B (upstream, FP8 quant) SGLang

qwen3.5-122b-gguf-q4km-llama-cpp

Qwen3.5 122B (GGUF, unsloth quant, Q4_K_M)

qwen3.5-27b-fp8-sglang

Qwen3.5 27B (upstream, FP8 quant) SGLang

qwen3.5-2b-bf16-sglang

Qwen3.5 2B (upstream, no quant) SGLang

qwen3.5-35b-a3b-bf16-sglang

Qwen3.5 35B A3B (upstream, no quant) SGLang

qwen3.5-35b-a3b-fp8-sglang

Qwen3.5 35B A3B (upstream, FP8 quant) SGLang

qwen3.5-397b-gguf-q3km-llama-cpp

Qwen3.5 397B (GGUF, unsloth quant, Q3_K_M)

qwen3.5-397b-gguf-q6k-llama-cpp

Qwen3.5 397B (GGUF, unsloth quant, Q6_K)

qwen3.5-4b-bf16-sglang

Qwen3.5 4B (upstream, no quant) SGLang

qwen3.5-9b-bf16-sglang

Qwen3.5 9B (upstream, no quant) SGLang

minimax-m2.7-awq4-vllm

MiniMax-M2.7-AWQ — AWQ quantized MiniMax M2.7

qwen3-coder-next-int4-autoround-vllm

Qwen3-Coder-Next-int4-Autoround

qwen3-vl-embedding-8b-vllm

Qwen3-VL-Embedding-8B — vision-language embedding model for multimodal retrieval

qwen3-vl-reranker-8b-vllm

Qwen3-VL-Reranker-8B — vision-language reranker model for multimodal retrieval

qwen3.6-27b-fp8-mtp-vllm

Qwen3.6-27B-FP8 — native FP8 format

qwen3.6-27b-fp8-vllm

Qwen3.6-27B-FP8 — native FP8 format

qwen3.6-35b-a3b-fp8-mtp-vllm

Qwen3.6-35B-A3B-FP8 — native FP8 format

qwen3.6-35b-a3b-fp8-vllm

Qwen3.6-35B-A3B-FP8 — native FP8 format

gemma4-26b-a4b

vLLM serving Gemma4-26B-A4B

glm-4.7-flash-awq

vLLM serving cyankiwi/GLM-4.7-Flash-AWQ-4bit with speed optimization patch

minimax-m2-awq

vLLM serving MiniMax-M2-AWQ with Ray distributed backend

minimax-m2.5-awq

vLLM serving MiniMax-M2.5-AWQ with Ray distributed backend

nemotron-3-nano-nvfp4

vLLM serving Nemotron-3-Nano-NVFP4 on a SINGLE NODE ONLY!

nemotron-3-super-nvfp4

vLLM serving Nemotron-3-Super-120B using CUTLASS kernels

openai-gpt-oss-120b

vLLM serving openai/gpt-oss-120b with MXFP4 quantization and FlashInfer

qwen3-coder-next-fp8

vLLM serving Qwen3-Coder-Next-FP8

qwen3-coder-next-int4-autoround

Qwen3-Coder-Next-int4-Autoround

qwen3.5-122b-fp8

vLLM serving Qwen3.5-122B-FP8

qwen3.5-122b-int4-autoround

vLLM serving Qwen3.5-122B-INT4-Autoround

qwen3.5-35b-a3b-fp8

vLLM serving Qwen3.5-35B-A3B-FP8

qwen3.5-397b-int4-autoround

EXPERIMENTAL recipe for Qwen3.5-397B-INT4-Autoround (please refer to README for details! Use with `--no-ray` parameter!)

minimax-m2.5

vLLM serving MiniMax-M2.5 with Ray distributed backend

qwen3.5-397b-a17B-fp8

vLLM serving Qwen3.5-397B-A17B-FP8