Speculative decoding with Gemma-4-31B + Gemma-4-E2B enables 120 - 200 tok/s output speed for specific tasks

So for my project I was using up until now either Gemini 3 / 2.5 Flash or Flash-lite. All my use cases are not agentic, simply LLM workflows for atomic tasks like extracting references from the law, classifying, adjusting titles to nominative case and so on. All this happens in non-English (LT) language, that's one of the reasons I originally used Google models, as multilingual quality is very great for small base languages. Each single request usually fits in 2k - 6k tokens context. Recently I

More from Reddit

Microsoft drops Copilot branding in Notepad for Windows 11 for everyone, but it’s really just a rename

Reddit

We translated the Palantir manifesto for actual human beings | What happens when an all-seeing eye looks in the mirror?

Reddit

New 3D device harnesses living brain cells for computing

Reddit

Training LFM-2.5-350M on Reddit post summarization with GRPO on my 3x Mac Minis — final evals and t-test evals are here

Reddit

Stanford researchers fed a language model a DNA sequence and asked it to create a new virus. It wrote hundreds of them, and 16 worked. One used a protein that doesn't exist in any known organism on Earth.

Reddit

You're right to push back.

Reddit