

7·
1 month agoThis isn’t really true — a lot of the newer MoE models run just fine on a CPU coupled with gobs of RAM. Yes, they won’t be quite as fast as a GPU, but getting 128GB+ of VRAM is out of reach of most people.
You can even run Deepseek R1 671b (Q8) on a Xeon or Epyc with 768GB+ of RAM, at 4-8 tokens/sec depending on configuration. A system supporting this would be at least an order of magnitude cheaper than a GPU setup to run the same thing.
This is false when it comes to me to PCIe, as mentioned elsewhere in this thread.
Most motherboards have cutouts on one end of the PCIe x1/x4 slots, for exactly this situation. If not, and you want to be adventurous, you can cut the plastic of the slot and it’ll work fine.
If the card is PCIe 3.0 x4, and the slot is PCIe 4.0 x1, the card will run at PCIe 3.0 x1. But it’ll work.