How we used gpt-4o for image detection with 350 very similar, single image classes.
This story recounts a challenging request that emerged in our small engineering team and how we solved it. The final solution demonstrates how LLMs have shifted what product-oriented teams can achieve with AI.
AskMona is a small company with an even smaller engineering team, striving to provide AI solutions to niche markets. We originally specialized in cultural institutions like museums and foundations but have since expanded into tourism and education.
One aspect of our product involves using computer vision to match points of interest—such as artworks in exhibitions—with personalized experiences. Due to our innovative reputation in the cultural field, a museum approached us with a specific request:
They had a large collection of car illustrations printed on exhibition walls and needed an app to match a picture of a car to its related content - identifier, image and links to more information. They also had a lean budget, so maximizing efficiency in infrastructure and maintenance was crucial.
The signature predated my arrival, but I know that our product team had initially planned to deliver an augmented reality experience via a web app, leveraging the image-based tracking abilities of this stack. This is not something we had in our main line of product, but the tech leader of the time felt this was the way forward. A specialized AR partner, established company in the field, was even identified to help us unblock any potential difficulties. The project was then shelved for later development as other matters took priority.
