LLMs accelerated with eGPU on a Raspberry Pi 5
After a long journey getting AMD graphics cards working on the Raspberry Pi 5, we finally have a stable patch for the amdgpu Linux kernel driver, and it works on AMD RX 400, 500, 6000, and (current-generation) 7000-series GPUs.
When I wrote about getting a Radeon Pro W7700 running on the Pi, I also mentioned AMD is not planning on supporting Arm with their ROCm GPU acceleration framework. At least not anytime soon.
Luckily, the Vulkan SDK can be used in its place, and in some cases even outperforms ROCm—especially on consumer cards where ROCm isn't even supported on x86!
Assuming you already have an AMD graphics card (I tested with an RX 6700 XT), and you built a custom kernel using our amdgpu patch (instructions here), you can compile llama.cpp on the Pi 5 with Vulkan support:
Note: I'd like to thank GitHub user @0cc4m especially for help getting this working, along with others who've contributed to the issues over on my Pi PCIe project!
