Here’s a game Claude Shannon, the founder of information theory, invented in 1948. He was trying to model the English language as a random proce

A Random Walk through the English Language

submited by
Style Pass
2021-06-13 21:00:07

Here’s a game Claude Shannon, the founder of information theory, invented in 1948. He was trying to model the English language as a random process. Go to your bookshelf, pick up a random book, open it and point to a random spot on the page, and mark the first two letters you see. Say they’re I and N. Write down those two letters on your page.

Now, take another random book off the shelf and look through it until you find the letters I and N in succession. Whatever the character following “IN” is—say, for instance, it’s a space—that’s the next letter of your book. And now you take down yet another book and look for an N followed by a space, and once you find one, mark down what character comes next. Repeat until you have a paragraph

Shannon was interested in the “entropy” of the English language, a measure, in his new framework, of how much information a string of English text contains. The Shannon game is a Markov chain; that is, it’s a random process where the next step you take depends only on the current state of the process. Once you’re at LA, the “IN NO IST” doesn’t matter; the chance that the next letter is, say, a B is the probability that a randomly chosen instance of “LA” in your library is followed by a B.

Leave a Comment