Phi-3-vision in 50 lines of C# with ONNX Runtime GenAI – nietras – Programming, mechanical sympathy, machine learning and .NET ❤.
Previously in Phi-3-mini in 30 lines of C# with ONNX Runtime GenAI I showed how easy it was to run the Phi-3-mini model locally in just a few lines of C#.
In this blog post I’ll show how to run the Phi-3-vision model, which is a multimodal model that supports text + image inputs, with .NET in a similar fashion with version 0.3.0-rc2 of ONNX Runtime GenAI based on the Phi-3 vision tutorial and phi3v.py.
Any of these directories can be used with ONNX Runtime GenAI. It does also mean there are large +2GB onnx.data weights file for each of these, and that you need to have each of these present if you want to run either of them depending on available hardware. The cuda-int4-rtn-block-32 is used here since I only have a RTX 3070 GPU with 8 GB VRAM.
