Public Petals Swarm: BitTorrent-style LLMs

submited by
Style Pass
2024-12-23 13:30:05

Contribute to the public Petals swarm and help deploy and fine tune Large Language Models across consumer-grade devices. See more about the Petals projecyt here. You'll get: - Eternal kudos from the community! - Access to all the models in the server - Easy access for inference (via Petals SDK and installation-free Kalavai endpoint).

Hardware requirements - 1+ NVIDIA GPU - 2+ CPUs - 4GB+ RAM - Free space 4x available VRAM (for an 8GB VRAM GPU, you'll need ~32GB free space in your disk)

Get the joining token. Visit our platform and go to Community pools. Then click Join on the Petals Pool to reveal the joining details. Copy the command (including the token).

Kalavai's pool connects directly to the public swarm on Petals, which means we can use their public health check UI to see how much we are contributing and what models are ready to use.

Models with at least one copy of each shard (a green dot in each column) are ready to be used. If not, wait for more workers to join in.

Leave a Comment