Vector Search Using OpenAI Embeddings With Weaviate
Lately, there’s been quite a buzz around ChatGPT and generative AI, so I thought I’d mess around a bit with OpenAI APIs. As I was experimenting, I found that the OpenAI Embeddings API is awesome, but it’s kinda been overshadowed by ChatGPT.
When we’re talking about vectors in the context of AI, they are used to represent unstructured data such as images or text. You can use vectors for ML tasks such as classification.
In this article, I will share how you can use the OpenAI embeddings API to easily create vectors and perform vector search on text data using Weaviate an open-source vector database. Let’s get started!
If you want to follow along with this tutorial you will need an OpenAI developer account (You need the API key). You will also need docker so that you can run Weaviate locally. Python version 3.8 or above should work fine for this tutorial.
By default, Weaviate does not persist data when run locally. We want to make sure our data is not lost even after shutting the database instance, so we will start the Weaviate instance with a volume. Open the docker-compse.yml file and add the volume as shown below. Save the yml file after changes.
.png)
