55

Ngram seems to be more authoritative than the Periodic Table here on EL&U. As someone with more than a passing interest in the language, I wanted to know how good Ngram is. And on Wikipedia, of all authorities to cite when seeking reliability, I found these relevant facts:

Point 1:

The Google Ngram Viewer or Google Books Ngram Viewer is an online search engine that charts frequencies of any set of comma-delimited search strings using a yearly count of n-grams found in sources printed between 1500 and 2008... in Google's text corpora in American English, British English, French, German, Spanish, Russian, Hebrew, or Chinese. ... generated in either 2008 or 2012

Point 2:

Google populated the database from over 5 million books published up to 2008. Accordingly, as of January 2016, no data will match beyond the year 2008, no matter if the corpora was generated in 2009 or 2012. Due to limitations on the size of the Ngram database, only matches found in over 40 books are indexed in the database; otherwise the database could not have stored all possible combinations.

Point 3:

The data set has been criticized for its reliance upon inaccurate OCR, an overabundance of Scientific Literature, and for including large numbers of incorrectly dated and categorized texts

From points 1 and 2 we learn that the English corpus contains 5,000,000 digitized books dating from 1500 to 2008. From point 3 we learn that there are criticisms. The overabundance of Scientific Literature is a particularly telling point in tracking words in common usage. This is important because this site itself shows us that usage in scientific literature is skewed and stilted: users often ask for a more polished word for something that seems to an educated person (shameless self-promotion there) to be perfectly normal and acceptable even within the ivied porticoes of academe. So the 5-cent word the questioner would have used at the dinner table could be lost to us, and the five-dollar substitute remains.

English is changing very rapidly, and I am sure that there has been as much change since 2008 as in the 30 years before that, in both UK and US usage. These changes will obviously not show up in the linguistic fossil record used by Ngram.

In Arabic, Chinese and Brazilian Portuguese there are corpora based on newspapers. These reflect current speech, are easy to update, and are accurately dated. Since much of the material is already digitized, there are no OCR problems. Now that there are so many publications online, it should be easy to construct one for any variety of English you choose, but I haven't found one so far. I reckon I need to look harder.

The article cited mentions that Ngram has been found useful in machine translation. I reckon that accounts for much of machine translation!

In view of all this, I have three related questions:

  1. In view of the fact that it is a fossil record, should we rely on Ngram as an authority for current usage?
  2. If Ngram is found to be wanting, what do we use instead?

(In reviewing the available tags, I was amazed to find there is none for Ngram, nor for current-usage as opposed to usage.)

  1. Is there an up-to-date currently-maintained corpus based on newspapers and Web sites for English?

migrated from english.stackexchange.com May 17 '16 at 20:00

This question came from our site for linguists, etymologists, and serious English language enthusiasts.

  • 2
    Fossil record? 2008 is a threshold for ancient history? There's a lot of vocabulary being created continuously (thanks, technology!) but it's not exponential growth. – Mitch May 17 '16 at 14:12
  • @Mitch English changes very fast these days. Just look at this progression: pox; venereal disease; social disease; STD; STI. Off the top of my head, that covers about 80 years of change. Or how about slow - retarded - learning-disabled - special? Granted, their meanings don't coincide but they do overlap, and the one that comes to your mind effectively dates you. BTW was the reference to exponential growth ironic? – frank May 17 '16 at 14:28
  • 1
    Not ironic. You said "I am sure that there has been as much change since 2008 as in the 30 years before that", I took that to mean rate same for 1/3 time range -> rate doubling -> exponential growth. Yes, language is changing and has a lot in the the 20th c, but I don't think the 21st century vocab additions are faster than 20th c. Also language change is more than vocab and I think syntax/phonology changin much less than formerly. – Mitch May 17 '16 at 16:41
  • 1
    @frank Why do you think English has changed more in the past 8 years than in the 30 before that? – Azor Ahai May 17 '16 at 16:43
  • 5
    1. Technology has advanced tremendously. 2. Media penetration is much greater than ever before. 3. The percentage of world population using English as a second language has increased. 4. Older authoritative voices such as the BBC in the UK and the 3 legacy networks in the US have given way to a plethora of new channels and media. 5. Schooling in the USA has become less normative. 6 Young people have more channels of communication, over a wider area, than ever before. 7. The Internet. All these tend to reinforce and potentiate each other, so we have many gas pedals and no brakes. – frank May 17 '16 at 16:50
  • Hasn't this issue been discussed before?, Why should we insist on it. As OP has clearly shown there is a huge amount of easily available material on the limits and possible drawbacks about using Google Books. Should it be a recurring theme we should regularly deal with? – user66974 May 18 '16 at 7:15
  • 3
    Possible duplicate of How accurate is Google Ngram as a language reference source? – JEL May 18 '16 at 9:06
  • 1
    @Josh61, you advocated moving this to meta and now that it's here you're saying it's redundant here? Or am I misunderstanding? – JEL May 18 '16 at 9:33
  • @JEL- I think this is the right place for this question, I think the issue has been discussed before. If users love it and and want to discuss further they can do it on this site. I didn't VTC this question here. – user66974 May 18 '16 at 11:40
  • 1
    @JEL I think you should consider undeleting your answer or providing it on the related question. It provides useful information that is not included in the other answers. – Kit Z. Fox May 18 '16 at 12:55
  • 5
    My comments are being deleted, once again, without warning, without reason. I mentioned that the fact five users voted to migrate the question to meta is not indicative that the majority of EL&U users agree. It is also undemocratic that contributors cannot vote to keep a question on the main site. Comments now deleted, not by me. Somebody tell me if they were vulgar, offensive or off topic please. – Mari-Lou A May 18 '16 at 19:44
  • 1
    Honestly, I find many statistics from Google to be inaccurate. So much so, I no longer use Google as a source, and never cite it. – RockPaperLizard May 19 '16 at 0:42
  • 8
    The primary value of Ngram to me isn't that it draws trustworthy lines representing the frequency of use of a word or phrase on a timeline; it's that it helps me search for actual occurrences of a word during a particular period of time. It doesn't do this consistently and reliably; but if you change the parameters systematically, you can find some interesting stuff that may not show up in a straightforward Google Books search. Flawed as it is, Ngram-based searches often find first instances of words and phrases that are older than the instances cited in OED. – Sven Yargs May 19 '16 at 1:04
  • 2
    One needs to note that the fossil record is rather heavily weighted in favor of animals that blundered into tar pits and the like. – Hot Licks May 20 '16 at 3:07
  • 3
29

Google Ngrams has problems for ELU questions:

  • it is not speech, which is often the unspoken context of questions, even though we only research things through print.
  • it is not the web, which has more colloquial usage (again more often the question context here)
  • it includes a lot of strange publications
  • it has a lot of OCR errors for older books (with messy printing, messy fonts)
  • it has a lot of metadata errors (especially date of publication)
  • it is not curated (it was a one-off in 2008)

Corpus of Contemporary American English (COCA) and British National Corpus (BNC) are somewhat comparable, curated corpora of newspaper English. They do not do websites. So the 'speech' (written language) is expected to be formal but up to date.

Many of the problems with searching Ngrams is not necessarily with the data but with the user's expectations of the search they attempt and therefore true of any corpus search. If you do a search on comparing the color terms pale vs light, there will be all sorts of FPs for multiple other meanings that will wash out entirely any kind of hypothesis support. There is some way around this (part of speech tags and phrases with surrounding words) but it can be difficult for any corpus and question. If you assess these results as a non-native speaker, you may very well get a blunt, misleading view of things, whereas a native speaker might understand the nuances of each instance better. So even with a (non-existent) perfect corpus, it may be difficult to justify definitive universal hypotheses (eg "'bucket' is always used in preference to 'pail' in a nautical context" (I made that up, I don't know)).

On the other hand, Ngrams is quick and easy, and gives links to the source immediately, so you can check your hypothesis with respect to context.

As to using an Ngram comparison on ELU as justification, you'd have to account for all the Ngram specific caveats plus the general word search problems.

So my suggestion is to, along with other online resources, use Ngrams but with great caution keeping in mind all the caveats. Don't rely on a single source. Corroborate with other corpora, a plain google search, talking to native speakers, comments here, chat here, etc, etc, etc. If you are a native speaker, you can be motivated by introspection of your own speech patterns but beware, simply asking a question or reading someone else's idea can bias you in a chaotically emotional direction.

  • 2
    Also indeed! I think – Matt E. Эллен May 17 '16 at 20:08
  • 1
    @MattE.Эллен I'm not going to put winning lottery numbers for tomorrow in plain text. Sprinkle with lemon water and hold over a light bulb until they appear. – Mitch May 17 '16 at 23:41
  • In the last five years since you have been a member of EL&U have you modified your opinion about this tool? Do you always steer away from using Ngrams? Have you had any negative experiences with using Ngrams? Could you share? – Mari-Lou A May 22 '16 at 12:26
  • 1
    @Mari-Lou this answer wasn't entirely balanced. Those are all rather caveats than commandments. I love google ngrams and it is the go-to first resource (so easy!) in first explorations. My opinion has increased favorably after discovering that the query language has POS tags (among others). As to negative experiences, the most negative is when it showed obvious supportable data that contradicted my hypothesis. That's the worst! – Mitch May 22 '16 at 12:44
  • 2
    @Mari-LouA Amended. How's that? Ngrams is not a table of logarithms, and it's also not Finnegan's Wake. Like everything, it's somewhere in between. I tried to put it on a relative scale (not as good as COCA but better than google hits), and also show that all such things, even when perfect, can be misused. – Mitch May 23 '16 at 15:45
  • Have you noticed that Frank, the OP, hasn't posted a thing since this question was moved to meta on May 17? Telling, isn't it? The question should have been left alone on the main site. We now have three questions on meta on this topic but none on EL&U. – Mari-Lou A May 27 '16 at 7:10
  • @Mari-LouA It's hard to tell how 'telling' it is. frank is assuredly pinged by activity here on meta (you're automatically a member of meta.X.SE when you join X.SE). Also, e can't see what he's not doing here. I don't disagree with you that this could have productively and on-topically remained on main, but it doesn't seem terrible that it is on meta either. – Mitch May 27 '16 at 12:38
19

One more problem with ngram that (I think) has not been mentioned (from Wired):

One of the traps in using ngrams to divine the popularity of people, ideas, or concepts is that a book only appears once—whether it’s been read once or millions of times. The Lord of the Rings is in there once, and so is some random paper on mechanics. The two texts are weighted equally. It doesn’t reflect what is people are talking about [or reading, or being influenced by] so much as what people are publishing about—and until very recently, most people didn’t have access to publishing. Like, what does this really tell you about language?

  • Actually this is a very important point. Can you show me the link? – user140086 May 17 '16 at 14:53
  • Excellent point! It's as if we are trying to count the population that once inhabited a city by counting the windows they passed by. Every window counts exactly once, whether it overlooked a main road or an alleyway. – frank May 17 '16 at 15:01
  • @Rathony Here you go: wired.com/2015/10/… – user66965 May 17 '16 at 15:06
  • Excellent observation, but it is true of absolutely every corpus ever. Reading is not language. If you want true language representation, you'd need a speech-to-text transcription of all utterances ever. – Mitch May 17 '16 at 23:37
  • 4
    Actually, one of the problems of Ngram is that it often does include multiple instances of the same work if it pulls the copies from different sources. Thus the Ngram for "of a certain age" for the years 1765–1795 yields ten matches, but two are links to duplicate copies. A search for the phrase "descend into a dingle" yields more than 40 exact matches—all from Ivanhoe. So with regard to a book's popularity skewing the Ngram results, Wired couldn't be more wrong. – Sven Yargs May 19 '16 at 0:52
  • 1
    @SvenYargs I think it could be more wrong - even if some texts are represented more than once, Wired's point is that the frequency of a term's occurrence in the ngram corpus doesn't accurately represent the frequency with which that term is used amongst speakers, and that point is correct. – user867 May 20 '16 at 0:08
  • 2
    @user867: You're right, of course, that Wired could be more wrong. In fact, after sober thought, I've concluded that my campaign slogan, if ever I run for public office, will be "I could be more wrong." – Sven Yargs May 24 '16 at 21:18
  • @SvenYargs: That seems likely to work better in moderator elections than those for actual public offices ;-) – SamB May 31 at 18:58
15

This was originally intended as an answer to the practical question of how current use of a term could best be gauged. As such, the answer lists common resources available for that purpose. Google Ngrams/Google Books do not satisfy the demand: being raw statistical data (the n-grams from the Google Books corpus) based on raw textual data (Google Books), inferences drawn from the mess are prima facie invalid and, if sound, only sound as a result of aleatoric serendipity; outside of tightly controlled experimental environments, the textual data as presented via the Ngrams interface is only useful as a biased and poorly focused selection of instances of term use during the stipulated time period.

Other than the painstaking, longwinded traditional lexicographical techniques of gathering and comparing dated uses of terms in context and over time, no technique or set of techniques (that I know of) suffices to address the desired distillation of 'contemporary use' into 'contemporary senses'.

I should stress that 'contemporary use', that is, what you call "current use", although I'm understanding it in the spirit you seem to intend (which is notably parochial), includes all historical use. That is to say, for example, that to the extent that Shakespeare is still read and discussed, Shakespearean usage remains contemporary to that limited extent, however rarely a Shakespearean sense of a still-common term may occur in popular contemporary use.

  1. In view of the fact that it is a fossil record, should we rely on Google Ngram Viewer as an authority for current usage?

See 2.

  1. If Google Ngram Viewer is found to be wanting, what do we use instead?

No (to question 1). Any "fossil record" should inform and condition conclusions regarding "current usage", but the notion that a fossil record is all that is presented when the beast (the literature) still walks the earth is just plain wrong. Google Ngram Viewer/Google Books has limited utility as a collection of corroborative evidence of term use to 2008. Current, that is, contemporary use, is best gauged with a wide-ranging combination of resources. For links to some of those resources, see the following.

  1. Is there an up-to-date currently-maintained corpus based on newspapers and Web sites for English?

Yes, if "up-to-date" and "corpus" are somewhat loosely used. The "corpus" is the sum total of World Wide Web resources available through general and specialized internet search facilities. "Up-to-date" for these resources is not "up-to-the-minute".

Aside from the obvious general internet search of contemporary journals and newspapers using any of a variety of search engines, free searchable newspaper corpora include the compilation from some US, Australian, New Zealand and Singapore newspapers at Elephind, and the more specialized compilation of US papers at Chronicling America. These each have their strengths and weaknesses, in terms of the results interface, etc. Elephind includes Chronicling America results, but the interface is weaker in some important areas (highlighting search terms, or not, being the most prominent area of disparity). Note also that Elephind includes some papers from as early as 1787 and as late as 2015 (the Elephind date range is a moving target because the Veridian enterprise is ongoing), while the Chronicling America collection is limited to papers published between 1836 and 1922.

Other useful sources of newspapers and other works include these:

  • The Hathi Trust Digital Library (contains many works also included in Google Books, and sometimes offers alternative routes of text access where access is limited via Google Books);
  • The Internet Archive and Open Library ("10,000,000 eBooks and texts");
  • Google Books (this is the base collection for Google Ngram Viewer; I find it returns different results than the Viewer, and so usually check both);
  • Veridian (many of these are included in Elephind results);
  • Free Newspaper Archives (links to various kinds of collections);
  • Newspapers+ Publisher Extra (some time in the past I gave up on the next paywalled source in favor of this one; I believe the problem with the next source was signal-to-noise ratio, but I don't remember, although I assume my reasons were sound);
  • Newspaper Archive (this bills itself as the "World's Largest Collection", but you'll have to pay to use it);
  • The Internet Library of Early Journals (limited collection of 18th and 19th Century English journals).

The usual specialized online corpora, such as COCA/COHA (Corpus of Contemporary/Historical American English), Micase (Michigan Corpus of Academic Spoken English), Corpus Concordance English, British National Corpus, etc. (see especially the listing shown at "CORPORA", BYU), should be mentioned. Some of these are listed in other questions and answers at the ELU meta site. (See the related questions and answers listed in a sidebar box associated with the linked question on the ELU meta site. The most comprehensive lists seem to be in What good reference works on English are available? on meta and What are your favorite English language tools? on main.)

For your specific question, there doesn't seem to be an organized and comprehensive answer; the questions and answers on ELU and ELU Meta referenced in my preceding paragraph are more general, very roughly organized, and don't include the newspaper or some of the other corpora I mentioned.

Raw Google Books n-gram data and more sophisticated interface

If you have the chops, the raw Google Books n-gram data is available for use. Unless whatever application you devise includes Google Books lookup and link collection facilities, of course, you will find the Google Ngram Viewer more convenient for many uses.

A much more sophisticated interface than the Google Ngram Viewer for the Google Books n-gram data is available via the BYU Corpora collection. See the interface comparison for details.

Up-To-Date Usage

For your question 3, two very useful corpora only recently (May, 2016) made available are the NOW Corpus of contemporary online newspapers and CORE: Corpus of Online Registers of English. For details see their respective sites, but here are brief descriptions from those sites.

For NOW:

The NOW corpus (Newspapers on the Web) contains about three billion words of data from web-based newspapers from 2010 to the present time. More importantly, the corpus grows by about 4 million words of data each day (from about 10,000 new articles), or about 120 million words each month.

With this corpus, you can see what is happening with the language this week -- not just 10 or 20 years ago. For example, see the frequency of words since 2010, as well as new words and phrases from the last few years.

For CORE:

This corpus contains more than 50 million words of text from the web. Unlike other corpora from the web, which are just big "blobs" of data, this is the first large web-based corpus that is carefully categorized into many different registers.

... You might pay special attention to the comparisons between registers and the (new) virtual corpora, which allow you to create personalized collections of texts related to a particular area of interest.

In addition to these recently released corpora, one of particular interest for researchers looking for information about contemporary informal usage is the Corpus of American Soap Operas:

The SOAP corpus contains 100 million words of data from 22,000 transcripts from American soap operas from the early 2000s [ed.: 2001-2012], and it serves as a great resource to look at very informal language.

  • 13
    The sum total of the World Wide Web cannot be searched reliably—Google hit counts cannot be trusted, and I don't know whether any better search engines for this purpose exist—and it is quite possibly more biased than Google Ngrams; it contains a large amount of text written by non-native speakers. – Peter Shor May 17 '16 at 11:14
  • 2
    Thanks for the copious and useful links to references, JEL. – frank May 17 '16 at 14:31
  • 4
    @PeterShor Indeed. 1) No search engine searches anything close to the total WWW. 2) Search algorithms are complicated, and more importantly they are the product that search engines are selling. They are not at all reliable for statistics except as a measurement of the algorithm itself. 3) Not only is there text from non-native speakers, there is also a lot of nonsense. Unlike published works, there is zero effort or oversight required to get some random text onto the WWW and into search engine results. – Era May 17 '16 at 17:28
  • @PeterShor, a good point which I thought I'd sidestepped with the "general and specialized search facilities" schtick. I see that hope was optimistic in this (meta) context. I'll try to clarify that my intention was to address a practical question, and raw statistics are worthless outside of controlled experimentation. – JEL May 17 '16 at 20:36
  • @PeterShor, Now that I look at it, bandaids aren't going to help. The answer was to a question, and was not intended as an entry in a somewhat random discussion. – JEL May 17 '16 at 21:19
  • 4
    Yay! Thank you for undeleting! Those links are necessary! – Mitch May 18 '16 at 19:40
6

Since the header asks broadly "How reliable is Ngram?" I thought it might be useful to point out a significant aspect of using Ngram as a search tool that casual users may not be aware of: the results change within the same period of years depending on what date range you set.

Ngram's default range is 1800–2000, but the tool permits you to set the endpoints to as earlier as 1500 and to as late as 2008. In my experience, however, the broader the date range you set beyond 1800–2000, the greater the number of matches from the default search will drop out of the Google Books search results available through the links beneath the Ngram graph.

For example, a search for "Jesus H Christ" for the period 1800–2000 yields (false positive) matches from 1804, 1805, 1817, and 1831, but the same search extended to the period 1500–2008 returns only the 1805 match from this group. On the other hand, shortening the date range for the search to 1800–1900 yields not only the original four matches but five additional (false positive) matches—from 1840, 1842, 1843, 1847, and 1853—not included in the the results returned for either of the other two searches.

From these examples, you might suppose that the more narrowly you define the time frame, the more numerous the results will be within that time frame. But that is not the case. A search limited to the period 1800–1870 provides no new matches compared with the 1800–1900 search, and a search limited to the period 1800–1865 yields matches only for the 1804, 1805, and 1817 false positives; inexplicably, all of the 1831–1853 matches vanish.

What this means in practical terms is that, if you want to use Ngram as a search tool to find texts published within a particular time frame of interest, you have to adjust the date parameters (sometimes repeatedly) to maximize the matches that Google Books returns. In the case of "Jesus H. Christ," the additional matches from the period 1840–1853 turned out not to be helpful, which is not surprising considering that the earliest publication year for any confirmed match for the phrase that I've been able to find is 1880. But in the course of this particular research effort, running multiple searches with different end years led me to focus eventually on the period 1860–1925, and Ngram's Google Books searches from that period yielded a number of highly relevant matches that did not appear in the search results for the default 1800–2000 time period.

  • 1
    Very nice find. Before anyone complains that 'the developers of NGrams suck', these kinds of selection problems on non-relational data have subtle differences like this. – Mitch Aug 24 '18 at 20:14
5

The fact is, NGrams can be a good tool for examining the language in published books. In fact, OED uses it as part of calculating word frequency! However, the results may not be accurate unless you go and verify they are.

OCR

The fact is, Google Books relies a lot on OCR. Unfortunately, there are a lot more OCR errors than just long s (which also confuses people who are not familiar with old typography). As this search indicates, it's also pretty common in older books to have a become o (and apparently, , become t). You'll also find that some books are very garbled, especially as you look further back in time. Even this book from 1880:

ran an 'm'ia 9w twz'mz n'n'rxz “'8 "BL E'BI'“! '8831.133 '9'61'66 'HJBHOBJ Yl'sz'vv az 'oz'v: n'n Lr'rlmz ru 51; "xx: we sn'uu Ht Lrn 13H fla-... “1 . 3 ;._:nm .ag_5 9's M'Isz wot 111st ar'st 1: nu rv'awo

This obviously obscures useful matches and may create false positives.

Words spelled the same

This is another pitfall I've seen people fall into. They'll try to compare two terms for frequency, and forget to consider matches that have other meanings (such as homographs) or matches that fall into another part of speech.

The opposite (when one word has many spellings) can become a problem in older texts (although you've usually run into other problems, mainly OCR, before this is an issue).

Korpus Vokabel

Yes, I may have specifically searched for English results here, but did you notice some (all?) of the actual results are in German? (Korpus and Vokabel are the German words for corpus and vocabulary.)

And if you can't trust "English" to only show results in English, I wouldn't trust "British English" to only return results in British English. For example, (looking at modern usage) this graph for color vs. colour shows one spelling is "preferred" in AmE and the other in BrE, but it also shows a number of results for the non-preferred spelling in each dialect (which doesn't seem accurate to me).

In general, there are some problems with metadata, which also affects the year published.

Part-of-Speech tagging

The logical solution to categorizing PoS is tagging (not that I've seen anyone else use this). According to the NGrams info page, part-of-speech tagging is usually accurate, but not so much for older English or uncommon usages:

The part-of-speech tags and dependency relations are predicted automatically. Assessing the accuracy of these predictions is difficult, but for modern English we expect the accuracy of the part-of-speech tags to be around 95% and the accuracy of dependency relations around 85%. On older English text and for other languages the accuracies are lower, but likely above 90% for part-of-speech tags and above 75% for dependencies. This implies a significant number of errors, which should be taken into account when drawing conclusions.

The part-of-speech tags are constructed from a small training set (a mere million words for English). This will sometimes underrepresent uncommon usages, such as green or dog or book as verbs, or ask as a noun.

???

There are some queries that inexplicably turn up the wrong results. One such query is this one, which shows a noticeable amount of usage for "bark up the" between 1805 and 1830. When you actually click on the search results, it is clear that there were 0 occurrences of the phrase during that period.

Statistics

This isn't actually a problem with NGrams itself, but nevertheless I feel it is relevant (as I've seen it become a source of error too). Many people seem to lack the statistics knowledge needed to understand some of what's involved with making and understanding NGrams. Luckily, their info page explains some of what you need to know:

Why do I see more spikes and plateaus in early years?
Publishing was a relatively rare event in the 16th and 17th centuries. (There are only about 500,000 books published in English before the 19th century.) So if a phrase occurs in one book in one year but not in the preceding or following years, that creates a taller spike than it would in later years.

Plateaus are usually simply smoothed spikes. Change the smoothing to 0.

What does "smoothing" mean?
Often trends become more apparent when data is viewed as a moving average. A smoothing of 1 means that the data shown for 1950 will be an average of the raw count for 1950 plus 1 value on either side: ("count for 1949" + "count for 1950" + "count for 1951"), divided by 3. So a smoothing of 10 means that 21 values will be averaged: 10 on either side, plus the target value in the center of them.

At the left and right edges of the graph, fewer values are averaged. With a smoothing of 3, the leftmost value (pretend it's the year 1950) will be calculated as ("count for 1950" + "count for 1951" + "count for 1952" + "count for 1953"), divided by 4.

A smoothing of 0 means no smoothing at all: just raw data.

Many more books are published in modern years. Doesn't this skew the results?
It would if we didn't normalize by the number of books published in each year.

However, I don't have all the answers. I'm still not sure what's the math behind comparing NGrams of different sizes, such as this (previously mentioned) one which compares a 3gram and 4gram.

1

If you use Ngram and just look at the graph you are looking at garbage. If you carefully examine a representative number of the actual references you get a much better idea as to whether the graph is meaningful or not.

Once you become a billionaire you may certainly create your own database (being sure to observe copyright limitations!) and set up your own site to search it. Until then I'm thankful that Ngram works as well as it does as often as it does.

  • I'm thankful that Ngram works as well as it does as often as it does is begging the question, not answering it. – frank May 17 '16 at 13:04
  • 3
    @frank, The ELU meta is regarded, spuriously, as a discussion site, rather than a question-and-answer site. Hot Licks' 'answer' is perfectly valid in that context. The meta site does, however, give short shrift to real, valid questions, supposing yours was one, rather than an effort to generate lists of resources. If yours was a valid question, it might be worth considering breaking it into chunks; as presented originally, it could be construed as "too broad" or "unclear" or another of the boogaboos to be found in the help center reasons for closing/migrating questions. – JEL May 17 '16 at 20:30
  • @frank - Your question asked "How reliable is Ngram?" I answered that question. – Hot Licks May 17 '16 at 22:38
  • @HotLicks, Frank, "Your question asked 'How reliable is Ngram?' I answered that question." <-- Surely you're having a giraffe? – Araucaria May 27 '16 at 23:57
  • I've upvoted this answer because it points to one of the most useful parts of Ngram - access to its corpus. As I understand it, Ngram provides search access to published material, which I'd expect to have a higher level of proofreading, grammar, etc than the pages found by simply trawling through the generic web. – Lawrence Jun 10 '17 at 11:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .