AMD Hipfire - a new inference engine optimized for AMD GPU's

Came across hipfire the other day. It's a brand new inference engine focused on all AMD GPU's (not just the latest). Github. It uses a special mq4 quantization method. The hipfire creator is pumping out models on huggingface. I don't know enough about quantization to know how good these quants are in terms of quality, but as an RDNA3 aficionado I'm happy AMD is getting some attention. Localmaxxing is a new LLM benchmarking site, and shows some pretty dramatic speedups for hipfire inference. Edit

More from Reddit

These Tiny Robots 50x Smaller Than a Hair Can Hunt and Move Bacteria

Reddit

AI swarms could hijack democracy without anyone noticing

Reddit

From game libraries to 8K video, Seagate rolls out drives up to 256TB

Reddit

Syria begins first trial of Bashar al-Assad and allies after his ouster

Reddit

Malian defense minister killed in attack as jihadi and rebel forces seized towns and military bases

Reddit

Apple CEO Tim Cook turned Apple into a $4 trillion juggernaut by not trying to be Steve Jobs

Reddit