qwen3.5-397b-int4-autoround

vllmray@eugr

EXPERIMENTAL recipe for Qwen3.5-397B-INT4-Autoround (please refer to README for details! Use with `--no-ray` parameter!)

Quick Info

Runtimevllm (ray)

Tensor Parallel2

Nodes2+

Dtypeint4

Containervllm-node-tf5

Tensor Parallel (--tp)GPU Memory (--gpu-mem)Port (--port)Served Model Name

sparkrun run @eugr/qwen3.5-397b-int4-autoround