Evaluating and enhancing probabilistic reasoning in language models
We strive to create an environment conducive to many different types of research across many different time scales and levels of risk.
Language models are capable of remarkably complex linguistic tasks. However, numerical reasoning is an area in which they frequently struggle. We systematically evaluate the probabilistic reasoning capabilities of LLMs and show that they can make more accurate inferences about distributions aided by the incorporation of real-world context and simplified assumptions.
Large language models (LLMs) have shown remarkable capabilities in understanding and generating text for a variety of linguistic tasks, including summarization of complex documents and zero-shot inference in specialist domains like medicine. At the same time, LLMs struggle with tasks that require numerical reasoning capabilities, such as calculating probabilities. Difficulties handling numbers may stem from the fact that most models rely on autoregressive next token prediction pretext tasks during training, which might not be suitable for mathematical operations, or simply because a limited number of numerical reasoning tasks are included in the model’s training corpora. Nevertheless, it is known that performance can be improved using prompt techniques, indicating that relevant knowledge may already exist within LLMs.
Probabilistic reasoning is a frequently used form of numerical reasoning that contextualizes samples within distributions. It allows people to not have to represent every detail of every sample that they observe, instead they can have the data summarized with a small number of parameters that describe the distribution. Understanding data distributions is important in many contexts. For example, in population health it can help determine if a person’s behavior is normative (e.g., is sleeping 8 hours unusual for a college-aged student?). In climatology, understanding temperature or precipitation distributions for a given day of the year at a particular location is important to determine whether or not an observation is typical or unexpected.
