Search code, repositories, users, issues, pull requests...
🌈 A Robust cloud-native AI Gateway - core LLMOps infrastructure stack for building production-ready AI Applications. It provides an Universal API for inferencing 100+ LLMs(OpenAI, Azure, Cohere, Anthropic, HuggingFace, Replicate, Stable Diffusion).
✅ Seamless Integration with Universal API ✅ Reliable LLM Routing with AI Router ✅ Load balance across multiple models and providers ✅ Automatic Retries with exponential fallbacks ✅ High availability and resiliency using production-ready LLMOps 🚧 Detailed Usage Analytics 🚧 PII detection and masking 🚧 Simple & Semantic Caching for cost reduction 🚧 No Vendor lock-in Observability - Logging, monitoring and tracing ✅ Enterprise-ready with enhanced security, reliability, and scale with custom deployments support.
Download the appropriate version for your platform from releases page. Once downloaded, the binary can be run from anywhere. Ideally, you should install it somewhere in your PATH for easy use. /usr/local/bin is the most probable location.
