Raspberry Pi 5 successfully accelerates LLMs using an eGPU and Vulkan
A Raspberry Pi 5 hooked up to an AMD Radeon-powered eGPU has been demonstrated using the graphics hardware to accelerate running a Large Language Model (LLM). Of course, it's Pi wizard Jeff Geerling again, and in the video embedded below, he talks us through his experience of leveraging the Vulkan API support to enjoy GPU-accelerated local AI on the Raspberry Pi 5.
In our last Raspberry Pi 5 connected to an eGPU progress report, we highlighted the modern AAA 4K gaming possibilities of this unlikely pairing. Games like Doom Eternal, Crysis Remastered, Red Dead Redemption 2, and Forza Horizon 4 were demoed running at 4K on our favorite $50 SBC. With most struggling to maintain performance above say 25fps, actual enjoyment of the titles would be another question.
Geerling ended his fun and informative video, last time, with an update on the Pi 5’s LLM support. He noted that he hadn’t managed to GPU accelerate any LLMs on the Pi 5, but smaller models could run on the CPU, in the Pi’s RAM. Moreover, with AMD basically ruling out ROCm support on Arm, prospects didn’t look good.
