Search code, repositories, users, issues, pull requests...
For detailed installation instructions of Nexa SDK for CUDA and AMD GPU support, please refer to the Installation section in the main README.
To update your current SDK version to v0.0.8.4, use the same command as the installation but add a -U flag to the pip install command.
You could use the following command to pull and run language models in GGUF format from 🤗 HuggingFace: nexa run -hf <hf model id>. Choose one of these commands based on your preferred model size:
The command line will prompt you to select one file from different quantization options. Use the number to indicate your choice. If you're unsure which one to choose, try "q4_0.gguf".
Visit the model pages to choose your parameters and quantization preference. We will constantly upload and support more models in the Qwen2.5 family.
