How Zapier builds production-ready AI products
Zapier is the world’s leading workflow automation platform, directly connecting to over 6,000 apps and supporting developers and non-developers at over 2 million companies.
Since then, the Zapier team has continued to ship amazing features like Zapier Central (AI bots), AI Actions, and many more internal and external use cases.
At Braintrust, we’ve had the opportunity to support Zapier since August of 2023. In that time, we've learned a lot by watching their AI development process evolve.
In this post, we’re excited to share some of Mike’s insights on how Zapier goes from an initial idea to a production-ready AI product:
Step 1 is all about quickly validating whether your AI feature idea is feasible with existing models. To do this effectively, you should very quickly cycle through different prompts and examples to get a sense for what works and what doesn’t. In this initial phase, Zapier only uses the smartest (and therefore the most expensive/slow) models - GPT-4 Turbo and Claude Opus.
After validating your idea, you should build and ship v1. Sub 50% accuracy is okay! In this stage of the development process, “vibes” (i.e. having your team sanity check outputs one by one) is sufficient to make progress.
