Language, Statistics, & Category Theory, Part 2
Part 1 of this mini-series opened with the observation that language is an algebraic structure. But we also mentioned that thinking merely algebraically doesn't get us very far. The algebraic perspective, for instance, is not sufficient to describe the passage from probability distributions on corpora of text to syntactic and semantic information in language that wee see in today's large language models. This motivated the category theoretical framework presented in a new paper I shared last time, whose abstract is shown on the right. But even before we bring statistics into the picture, there are some immediate advantages to using tools from category theory rather than algebra. One example comes from elementary considerations of logic, and that's where we'll pick up today.
Last time we defined a category of language $\mathsf{L}$ whose objects are expressions in English (strings of words) and where there is a morphism between two expressions if one is contained as a substring of the other. This category is pretty bare-bones; it just tells us "what goes with what," a bit like syntax. To glean something about semantics we then passed to a better category $\mathsf{Set}^\mathsf{L}$ whose objects are functors $F\colon \mathsf{L}\to\mathsf{Set}$, which are just ways to assign a set to each expression in language in a way that's compatible with the substring containment of those expressions. Explicitly, a functor $F\colon \mathsf{L}\to\mathsf{Set}$ is an assignment of a particular set $F(y)$ to each expression $y$ in language with the property that if $y$ is contained as a substring of $z$, then $F(z)\subseteq F(y).$ Such functors are also called copresheaves, though let's just call them "functors" for now.
.png)
