Amazon adds Statistically Improbable Phrases

In their quest to add more and more arcane content to every page, Amazon recently added Statistically Improbable Phrases to their pages for books that have the “Search inside…” feature. Apparently, Amazon is using an algorithm to determine which phrases in particular books are less likely to appear in other books with some interesting, though not terribly useful, results. Or so it would seem to me. (Although there is the prospect of a third party using this data to come up with some interesting applications). Anyway, to see it in action, let’s look at the page for Oblivion by David Foster Wallace, and you’ll see this near the top of the page: ” SIPs: consultant caste, executive intern, snoring issue, head intern, dominant village,” those, apparently, being some of the Statistically Improbable Phrases contained within the book. Then, if you want you can click on one of the SIPs to see other books that contain it. Here’s the short list of books that contain the phrase “snoring issue.”