Search code, repositories, users, issues, pull requests...
Chatty is your private AI that leverages WebGPU to run large language models (LLMs) natively & privately in your browser, bringing you the most feature rich in-browser AI experience.
This project is meant to be the closest attempt at bringing the familarity & functionality from popular AI interfaces such as ChatGPT and Gemini into a in-browser experience.
By default, WebGPU is enabled and supported in both Chrome and Edge. However, it is possible to enable it in Firefox and Firefox Nightly. Check the browser compatibility for more information.
The Dockerfile has not yet been optimized for a production environment. If you wish to do so yourself, checkout the Nextjs example
To run the models efficiently, you'll need a GPU with enough memory. 7B models require a GPU with about 6GB memory whilst 3B models require around 3GB.
Chatty is built using the WebLLM project, utilizing HuggingFace, open source LLMs and LangChain. We want to acknowledge their great work and thank the open source community.
