Reimagining database querying on unstructured data

submited by
Style Pass
2021-08-28 15:30:07

By organizing information, databases are an essential component of nearly every computer program and online service. But the rigid structure of conventional database systems also constrains how they can be used. These systems require preset schemas and can only answer queries with well-defined semantics written in SQL (structured query language). Queries must be exacting to return correct information. Moreover, the data must be stored in a way to comply with the schema; therefore, taking advantage of the abundance of available unstructured data is challenging.

Facebook AI has developed a new approach called neural databases, which enables machines to search unstructured data — which might range from vast collections of text to recordings of songs — similar to how traditional systems can search a typical structured database. With neural databases, it might one day be possible to run a complex query such as “What is the third-longest entry about a Russian-born novelist?” directly on Wikipedia, for example.

Neural databases bridge an important gap between the fields of databases and NLP. Significant progress has been made in using natural language queries on standard structured data. This lets people pose ad hoc queries such as “How many teams won away games by more than three points?” But these existing systems can’t query a collection of information that isn’t organized into a structured database. Conversely, machine learning models can provide powerful predictions for tasks whose semantics are vague and that involve data that does not fit into a predefined schema. “Does this post contain hate speech?” for example. However, they do not have the benefits of composition that databases possess. Therefore, it is difficult or impossible to extend them to closely related but unseen predictions — such as “What percentage of reviews are positive for horror movies released in the 1970s?” Or even “How many directors under the age of 30 released positively reviewed horror movies in the 1970s?”

Leave a Comment