6

This is a constant issue on ELU from the very beginning but one that I feel has never been explicitly addressed.

Many questions and corresponding answers hinge on how well known a word is.

For example 'chattel' (a cognate of cattle meaning livestock or people treated as material goods) is not a top 1000 word in English, yet it could be expected to be understood in context by someone with a (recent) college degree (in the humanities). But to say "'chattel' is a common word" is very misleading, as I'm sure many people might not recognize it. But it is way more common than say 'adze' (a hoe-like farming implement).

What I am looking for is an economical but objective way to say how common a word is. Number of hits in a corpus like COCA would be a good start, but it kind of needs a legend to go along with it (other words with their frequencies to show relevant commonness).

Is there some kind of scale that a word can be placed on that objectively says how common a word is and is easily understandable? Something like 'basic - top 1000' '1st grade - top 2000' 'newspaper - top 10K' 'economist - top 20K' 'scrabble player - top 100K' with examples? Or the word levels in freerice.com? ghits (with all its problems)?

We may want to make a FAQ page for this for reference.

Basically we want a Zipf curve for English words and labels (and exemplars) for percentiles (or gross categories). Is there something like this that exists already somewhere?

13
  • Yes, I have done a google search, and there's lots on Zipf curves in general, but nothing that also gives an indication about what it means to be word #3,798. Is that an obvious word, obscure, 10th grade, 2nd grade or what? (yes, probably 2nd grade since most adults have about 20K vocab). But maybe my google-fu ain't so great.
    – Mitch
    Jun 5, 2015 at 17:12
  • 2
    For what kind of questions does the commonness of words matter?
    – curiousdannii
    Jun 5, 2015 at 22:56
  • @curiousdannii Many questions about word usage would be made much more useful if we could tell the asker that a word, though it exists, is very rare, or that word X is much more common than word Y. But for literally questions about 'how common', recent ones are boondoggle, thrice, gesundheit
    – Mitch
    Jun 6, 2015 at 1:44
  • questions which are literally "how common is X?" should be closed for lack of research. If they're asking about origin too, then the commonness should be edited out of the question.
    – curiousdannii
    Jun 6, 2015 at 1:59
  • 1
    @curiousdannii lack of research? not everyone knows about COCA. More people know about etymonline than COCA. But commoness is a good idea of how to use a word. If it's very uncommon then it will probably be very formal or technical or archaic, and a frequency is a quantitative way of saying that.
    – Mitch
    Jun 6, 2015 at 2:10
  • Just because you're not sure how to research something doesn't mean you get to escape the off-topic standards.
    – curiousdannii
    Jun 6, 2015 at 2:11
  • 1
    @curiousdannii you're implying that something is off-topic here. Can you elaborate?
    – Mitch
    Jun 6, 2015 at 14:08
  • 1
    I have searched for and found numerous lists of words by frequency, but as yet no labels for the sections beyond 'basic' English for first 1000.
    – Mitch
    Jun 6, 2015 at 20:33
  • I guess it's not strictly off-topic, but anything with a lack of research can be closed with the non-enough-research close reason. Simply asking how common a word is definitely counts as not enough research.
    – curiousdannii
    Jun 6, 2015 at 23:30
  • Perhaps it would help if we had a Meta FAQ post listing sites which will tell you how common words are, linking to Wordfrequency.info and Google NGrams etc. (Maybe there's one already that we can link to when these questions come up.) (Haha, I just noticed you suggested an FAQ in the post itself.)
    – curiousdannii
    Jun 6, 2015 at 23:44
  • 2
    @curiousdannii I agree that just asking how common a word is isn't a great question, but information about how common a word it could still be an important part of a good answer to a different and less trivial question.
    – user867
    Jun 9, 2015 at 6:58
  • It's fine if you want a convenient way for people to find facts about word frequency. It's not fine if you want to impose your own peculiar ideas about word frequency on others. Personally, I doubt if this is ever of any importance in solving problems people have in understanding how English works, but I would have no interest in imposing my view on you.
    – Greg Lee
    Jun 10, 2015 at 16:57
  • Notice that "Scrabble player" would be a very skewed representation of large vocabulary; top Scrabble players will know all of the two and three letter words, at least most of the four letter words, thousands of seven and eight letter words corresponding to the most likely tile combinations, words heavy with vowels or consonants, and most of the JKQXZ words. All of these are useful in Scrabble, but not necessarily a useful general vocabulary, and on top of that they often only know which words exist, and not what they actually mean.
    – Peter Olson
    Jun 14, 2015 at 0:24

2 Answers 2

Reset to default
4

A lot of good answers I see on questions like those referred to link to a query in the Google Ngram viewer, such as this link on this question. This viewer shows not only the relative frequency of words, but how that's changed over time.

With a fairly low cutoff, Randall Munroe published a book explaining challenging concepts using only the most frequent 1000 words. There's also a text editor here which lets you easily check your own writing against that list, though the source of that list isn't quite clear.

3
  • 1
    Google Ngrams has ... difficulties ... like with reliability... or with sources. It is very informative but one must beware (a lot of its text is OCR'd which is terribly problematic).
    – Mitch
    Jun 6, 2015 at 1:37
  • But it is really good at relative frequency and time. But It'd be nice to be able to answer with absolute frequency 'Yes, well known word', 'no, very uncommon word'.
    – Mitch
    Jun 6, 2015 at 1:37
  • Yes, Munroe's book is another motivation in the cloud of ideas surrounding this request. When someone claims here that everyone should know the meaning of X, we'd be able to back up that claim with 'It's one of the 1000 most common words and anybody with 2 years of ESL should know it.' vs 'No, that word is the 40,000 most frequent word and since most native speaking adults have a vocabulary of 30K, most people would not even recognize it' (or something to that effect).
    – Mitch
    Jun 6, 2015 at 20:31
3

Wordfrequency.info claims that it contains the most accurate frequency data of English.

Our data is based on the 450 million word Corpus of Contemporary American English (COCA) -- the only large corpus of English that is both up-to-date (the latest texts are from Summer 2012) and which is based on a wide range of genres (e.g. spoken, fiction, newspapers, magazines, and academic writing).

2
  • nice...that is derived from the COCA corpus. Do you know if they have a categorization of the words from beginner to scrabble player?
    – Mitch
    Jun 5, 2015 at 18:52
  • @Mitch: I didn't have a chance to check in detail.
    – ermanen
    Jun 5, 2015 at 21:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .