Will llama.cpp multislot improve speed?

I've heard mostly bad opinions about multiple slots with llama.cpp (--parallel > 1). I guess comparing to vLLM it might be worse at this, but I recently tried vLLM on 4 slots and it indeed improved the overall speed significantly (150-170tps decode on one slot llama.cpp to 400tps with 4-slot vLLM, of course when all 4 slots are used). BUT vLLM handles CPU offload poorly (or I don't know how to use it properly) and, from what I heard, doesn't work with GGUFS too good, and thus, limits the availab

More from Reddit

Stanford researchers fed a language model a DNA sequence and asked it to create a new virus. It wrote hundreds of them, and 16 worked. One used a protein that doesn't exist in any known organism on Earth.

Reddit

This is where we are right now, LocalLLaMA

Reddit

DeepSeek V4 Update

Reddit

"Weights are coming".Xiaomi’s MiMo V2.5 Pro has landed at 54 in the Artificial Analysis Intelligence Index.

Reddit

Sam Altman May Control Our Future—Can He Be Trusted?

Reddit

Pink Floyd’s The Wall As A Cartoon Network TV Movie

Reddit