Making SQL Smarter: How to Embed AI into your Queries with HeavyIQ
We’ve been hard at work over the last year to bring native conversational AI capabilities to the HEAVY.AI platform as part of our HeavyIQ module, primarily focused to date on using locally hosted, custom trained LLMs to allow users to generate queries and visualizations with natural language.
However there are many use cases which call for deeper embedding of Conversational AI into the analytics pipeline. What if vanilla SQL queries could be augmented to handle use cases which require the sort of “fuzzy intelligence” that humans excel at, but are difficult to write explicit logic for? ELT use cases like wrangling poorly formatted dates, sentiment analysis, entity extraction, and even basic fact augmentation are difficult if not impossible to do with standard SQL but constitute sweet spot use cases for LLMs.
To this end, we’ve added the ability to call the HeavyIQ LLM directly from SQL itself with a new LLM_TRANSFORM operator, which can be a huge time saver for use cases like the above. To make this work, we've trained the HeavyIQ Language Model to be able to tersely answer a wide variety of questions, and we even support grammar constraints in the form of allowed outputs and regexes to ensure that the output is readily usable downstream for analysis. On the performance side of things, we ensure we call the LLM in parallel across many threads to yield the best possible throughput, and we only make a single call per unique input to avoid redundant processing (for example, calling LLM_TRANSFORM on a column with a 1,000 unique values over a billion rows will be fast as the LLM only needs to be queried 1,000 times).
