(Linux) Has anyone succeeded in using NVMe space as substitute RAM for larger models? Is it worthwhile?
·
0 reactions
·
0 comments
·
0 views
So I have a consumer-grade AMD GPU with 24gb VRAM and 64gb DDR5 RAM which have served me well enough for models up to around 120B. Of course, this just isn't enough for larger models in the 300B+ range. Storage and RAM are expensive so I'm not going to be upgrading my hardware any time soon, but I have plenty of high speed NVMe space available. Is it possible to leverage this as a workaround? What would be the method, swap file? Do I need to take any special steps to make sure something like lms
Original article
Reddit
Anonymous · no account needed
