34

This may be a stupid question to ask, and maybe I will be chastised for not doing the research myself, but recently I have been having doubts on the validity of Ngram charts. Let me explain where my misgivings lie.

First, in the American corpus, does it contain only American publications. Would a Charles Dickens novel, for example, never show up in its results?

Second, how reliable is Ngram for showing the difference between AmEng and BrEng spellings and language usage? For example, I saw these charts in tchrist's posted answer to How and when did American spelling supersede British spelling in the US?

enter image description here enter image description here

Well, did tires or tyres (as in auto-mobile wheels) really exist in 1800? What I believe we are seeing is the verb, to tire. (EDIT: Many thanks to @Mario Elocio who pointed out that tires existed in 1800 but were made of iron, not rubber.) And how reliable would the chart be in any case if we typed humor vs humour if Google books included quotes spoken by American and British speakers in novels?

I have the same doubts concerning the British corpus, would a J.D. Salinger novel or even a quote from "Catcher in the Rye" never appear in the British corpus results? If books are published in both countries which have different copyright laws (if I recall) then wouldn't both corpora include those texts?

Finally, if one wanted to find out whether a recently coined expression or even an old slang expression is spoken more frequently in one region than in another e.g in the North of England compared to the south west, how would one go about researching it? Wouldn't Ngram be useless in this case? The same question could be said for "y'all" vs. "you guys" I keep reading that the former is usually confined to the southern states of the US whereas the latter is more commonly said elsewhere in the States. Maybe after the phenomenal success of Breaking Bad the TV series, this usage is changing and is spreading across all the states and even crossing the pond as I write this piece. How would I know, if I am living in Italy? :*(

EDIT On ELL @Matt has posted this Ngram on his answer to: What is a word for “very slow”?

enter image description here

I think this is a perfect example that illustrates that Google Ngram is unreliable for certain tasks. This diagram cannot illustrate whether one expression is more widely spoken than another. What it seems to be telling us is that the expression "glacial pace" has recently become more frequent. Despite Matt having changed some of the searched words (this is version B), I still don't understand the scope of this Ngram. Am I missing something?

26
  • 3
    No, sorry, actually I meant to show that NGrams does refer to a thing apart from Google NGram Viewer.
    – Ramchandra Apte
    Nov 17, 2013 at 11:12
  • 4
    Interesting question, but how does it concern the English language? Suggest migrating to meta.
    – Tim Lymington
    Nov 17, 2013 at 12:26
  • 4
    @TimLymington I think it is very connected to ELU, because many users use Ngram to back up their answers and provide evidence. Furthermore, if the question was about The Oxford English Dictionary, would you still think it off-topic? Dictionaries and language go hand in hand. I see Ngrams as another reference tool.
    – Mari-Lou A
    Nov 17, 2013 at 12:31
  • 1
    This question appears to be off-topic because it is a better fit on meta.
    – Autoresponder
    Nov 17, 2013 at 15:03
  • 1
    I'd prefer this question not to be migrated. I think it is more useful for users, especially inexperienced ones. Not everyone visits meta.
    – Mari-Lou A
    Nov 17, 2013 at 16:16
  • 1
    Perhaps, but I would think the examples would be very rare and numerically insignificant.
    – Mari-Lou A
    Nov 17, 2013 at 16:30
  • 4
    @Mari-LouA I agree, it should not be migrated, since many use Google ngram for statistics and mentions. BTW your AmE vs BE tire/tyre examples are interesting, but a tire was originally the iron rim on a wheel. Possibly short for attire.
    – Mario Elocio
    Nov 17, 2013 at 17:05
  • 2
    @Mari-LouA Yes they are! Maybe I should have phrased my comment differently, re your question about tires/tyres in 1800. Yes there wire tires in 1800, only they were iron tires, not rubber tires
    – Mario Elocio
    Nov 17, 2013 at 17:21
  • 2
    I didn't know that. I've learnt something new today, thanks!
    – Mari-Lou A
    Nov 17, 2013 at 17:23
  • 3
    Related: meta.english.stackexchange.com/questions/2469/…
    – Kit Z. Fox Mod
    Nov 18, 2013 at 1:32
  • 2
    @KitFox Very informative, thought provoking and interesting read. One could now argue that this question is a duplicate. I still maintain that this question or better still, Robusto's, should be present on ELU and not on meta. On meta we are preaching to the chorus. While searching for questions related to Ngram, I found nothing that answered my question. If I had thought of posting here I would have seen his and read the answers. Although there is no mention of orthographical and colloquial differences between BrEng and AmEng, which was the motivating force behind my post.
    – Mari-Lou A
    Nov 18, 2013 at 6:52
  • 3
    Note that you can tag words for part of speech: tire_NOUN versus tyre_NOUN for the example above. There are some other interesting tags too.
    – Bradd Szonye
    Nov 18, 2013 at 22:07
  • 2
    Also note that for this particular example, you get better results searching for the plural, tires versus tyres.
    – Bradd Szonye
    Nov 18, 2013 at 22:15
  • 3
    One major problem with Ngram Viewer is how it handles -n't. For example, if you tried to compare can't with cannot and can not, all of which are distinct in English, you'd end up with a single line rather than three.
    – user28567
    Nov 21, 2013 at 18:34
  • 3
    Possible duplicate of Should we allow Google NGrams to be presented as statistical evidence without qualification? Should we define a set of standards for their usage?
    – Robusto
    Mar 29, 2018 at 23:10

2 Answers 2

Reset to default
12

  • re AmE vs BrE: Dickens is published in the US and UK. Most popular authors will be in both. Check out the ngrams info for the source of the publications.

  • re speech vs print: ngrams only captures those things that have been printed in books, not transcribed speech. So lots of spoken slang, nuances if pronunciation, regional varieties are sparsely represented and even when so have all sorts of orthographical issues.

  • re accuracy (which is really your question) grams is a tool that works very well...as all tools it can be misused. It is not telling falsehoods but it may be misunderstood. It will search for the thing you stated. But that may not be the right thing to look for. For example, if you compare "y'all" and "you all", the latter may find appearances of "... you. All ...", it may even find you followed by all which are in different constituents "The bank wiill not give you all the cash". So you have to be very careful about the question you're asking

  • actually there may be 'difficulties' (I hesitate to say bug) with the encoding of the original texts and the meta data. It is well know that the meta data can have all sorts of wrong things, wrong dates, wrong authors, OCR errors, etc etc etc，so....

The advice is to use ngrams with caution and always look atthe actual results rather than just blindly following the graph results.

NGrams is a great tool but it shouldn't be used blindly.

6
  • Thank you! Especially for confirming that classic novels are found in both corpora. I was wondering about that bit.
    – Mari-Lou A
    Nov 17, 2013 at 16:11
  • 3
    I think most classics are in the public domain and so easily published by any body. Also most publishers have international offices so that once a book becomes popular at home, the foreign office gets to publish it too(eg Harry Potter). I think most differences are from rarer books and journals. Look at the individual hits in any return and you'll see.
    – Mitch
    Nov 17, 2013 at 16:17
  • 3
    There are lots of problems with metadata and faulty scans, as you say. I'm not sure whether different prints of the same book are categorised as falling under different corpora; can you prove this? When I tried to find a random phrase from Oliver Twist, I got zero results! Try and search for after telling Oliver that she in Ngrams. There are many things that are wrong with Ngrams in inexplicable ways. Lastly, I do believe it is supposed to take punctuation into account; remember that the results you see in Books are not what Ngram has indexed: it's just a parallel search.
    – Cerberus - Reinstate Monica
    Nov 18, 2013 at 1:58
  • 'Dodger and Charley Bates entered' works, though: Ngram has results for this phrase from O.T. both in the British and in the American corpora. Those are probably not all found in excerpts or quotations.
    – Cerberus - Reinstate Monica
    Nov 18, 2013 at 2:01
  • 1
    "you all" won't match "you. All" because Google Ngrams is case sensitive. There are a lot of OCR problems with Google Books, though. Remeber that a search in Google Books is not the same as a search in Google Ngrams.
    – Matt E. Эллен
    Nov 20, 2013 at 11:52
  • @Matt if anything that error shows that NGrams is not obvious, making my point (unintentionally).
    – Mitch
    Nov 20, 2013 at 13:38
3

The tire / tyre example would be heavily confounded by tire as in become tired, and to a tiny extent by places in Lebanon and New York (wp), as well as the iron tyres mentioned in the comments.

So that serves as an example of how ngrams can easily fool or be fooled.

2
  • 3
    But note that we can give the part of speech like this: tire_NOUN (ref). Although it should be noted that there might be POS tagging errors, but in general this helps, since the state-of-the-art accuracy of automated POS tagging is quite high. =)
    – justhalf
    May 6, 2014 at 9:26
  • I found this discussion very interesting. I am wondering whether there have any changes to the functionality of NGram since the OP.
    – Livrecache
    Oct 4, 2017 at 0:27

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .