While the model can imitate the style of proving this simple theorem to some extent, there is still a huge gap with human-level accuracy. Completion o

GPT-J-6B: 6B JAX-Based Transformer

submited by
Style Pass
2021-06-09 05:30:07

While the model can imitate the style of proving this simple theorem to some extent, there is still a huge gap with human-level accuracy.

Completion on a question from BoolQ (SuperGLUE). While both sampling methods result in the same correct conclusion, the nucleus sampling hallucinates and contains incorrect reasoning, while the greedy sampling answers concisely and reasonably. In general, we observed that greedy sampling is more accurate and contains less hallcinations than nucleus sampling when the output is supposed to be short like this, which is predictable given that classification task is usually done with greedy sampling.

Leave a Comment