Introducing diversity in online language analysis

For the past 30 years, computer science researchers have been teaching their machines to read standard English — for example, by assigning back issues of the Wall Street Journal — so computers can learn the English they need to run search engines like Google. But using only standard English has left out whole segments of society who use dialects and non-standard varieties of English, and the omission is increasingly problematic, say researchers. …read more