Self-hosting AI with Spare Parts and an $85 GPU with 24GB of VRAM
An inexpensive AI server using a Nvidia Tesla M40 GPU, Proxmox VE, Forge, Oobabooga, remote access via Tailscale, and some leftover spare-parts.
I have been running Stable Diffusion with some success at home for quite some time thanks to the AUTOMATIC1111 fork from lshqqytiger . I’ve used it to generate images to complement some of my content creation. But as a Windows 11 user with a Radeon 7900XTX GPU, I’ve learned that this combination can be equal parts frustrating and disappointing.
Like many other people in this space, I’ve been fascinated with Flux.1 from black forest labs. I used Flux.1 to generate a few images for the Topton DIY NAS motherboard rundown blog that I wrote for Butter, What?! and knew immediately that I wanted to leverage Flux in my content creation, but I couldn’t yet run Flux locally thanks to my combination of operating system and GPU.
Both of my DIY NAS and homelab servers are small form factor, which means they lack enough available PCIe slots and the physical space in the case to fit a full-size GPU, which made moving my AI workflow off my desktop computer challenging. In early August, I posted a poll about my AI predicament on Patreon with a few options for my AI workflows in mind. Ultimately, I didn’t like any of those options. They seemed expensive, the benefits of their outcomes were questionable, and as a result I wasn’t certain I’d get value out of any of them. At the conclusion of that poll, I decided I was not willing to disrupt my status quo.
