Lydie.cc — Solar AI/LLM Server V2
Meet SolarAI LLM Server V2, now complete. Probably the world's first quad GPU, DDR3 based rig using HDDs for LLM storage that generates 33.4 tps with the Strata QWEN3.8-120b model, and it cost me about a grand USD.
This is the hardware/software stack:
- Asus Z9PE-D16 motherboard
- CPU: 2x Intel Xeon E5-2620 V2 Processor
- GPU: FOUR NVIDIA Tesla P100 16GB PCIe (64GB VRAM)
- RAM: 128GB DDR3 - 16 sticks
- Storage: 120GB SSD for OS
- Storage: FOUR striped 500GB WD Velociraptor HDDs for models
- OS: ubuntu-24.04.5 server amd64
- CUDA 12 for Pascal
It will run my Hermes/Home Assistant, and is not used to generate slop.
Performance & Power:
- Runs strata-swift-iq3_xxs
- Initially achieved ~33.4 tps, which is impressive for a DDR3 platform and a 120B model!
- Power Consumption: Idle: ~200W / Load: Pushes 700W+ across the four GPUs during inference.
Cost & History:
- Budget: The build cost approximately one grand USD out-of-pocket.
I asked the Hermes Agent running inference on this server to make a website about itself: https://lydie.cc/solaraiv2
