Search code, repositories, users, issues, pull requests...

Kalavai is a self-hosted platform that turns everyday devices into your very own AI cluster. Do you have an old desktop or a gaming laptop gathering dust? Aggregate resources from multiple machines and say goodbye to CUDA out of memory errors. Deploy your favourite open source LLM, fine tune it with your own data, or simply run your distributed work, zero-devops. Simple. Private. Yours.

To create an AI cluster, you need a seed node which acts as a control plane. It handles bookkeeping for the cluster. With a seed node, you can generate join tokens, which you can share with other machines --worker nodes.

The more worker nodes you have in a cluster, the bigger workloads it can run. Note that the only requirement for a fully functioning cluster is a single seed node.

Once you have a cluster running, you can easily deploy workloads using template jobs. These are community integrations that let users deploy jobs, such as LLM deployments or LLM fine tuning. A template makes using Kalavai really easy for end users, with a parameterised interface, and it also makes the platform infinitely expandable.

Leave a Comment