LLMs are not even good wordcels
A chat with friends recently reminded me about pangrams, and what a cute little language curiosity they are. I also remembered that i never got a self-enumerating pangram generator to work. I should give that another try! I thought it would be fun play with ChatGPT and see if it could generate some good ones. I expected it to do very well on this task. After all, LLMs should be excellent wordcels, right? Yeah, i know the “wordcels vs. shape rotators” meme is kinda cringe, but i find these terms ironically endearing. Well, it doesn’t seem so.
Just so we’re on the same page: pangrams are phrases that use every letter of the alphabet. Even if you didn’t know that definition, chances are you already knew of one such phrase: “the quick brown fox jumps over the lazy dog”, which is typically used to showcase typographic fonts.
I went with Spanish because, why not? It’s nice when software understands your native language. In fact, maybe even programming languages should be designed to let people program in their mother tongue. Learning how to program when you’re not fluent in English, or when your alphabet is not based on Latin, is more difficult and frustrating than it needs to be. A recent CoRecursive episode touches on this idea, and i found the guest’s arguments to be very compelling. She created Hedy, a language that allows students to learn programming using their native tongue. The breadth of languages it supports, from Spanish to Korean, is awe-inspiring :) And, in any case, if it were true that LLMs have an internal model of the world, the output language shouldn’t make much difference.
