For some time now, contributors to EL&U have offered NGrams in support of their arguments. Now, there is nothing wrong with this practice per se: I have done so myself, and have seen others do it in a way that acknowledges the margin for error inherent in a flawed system. When done well it is done in a spirit of inquiry, citing the NGram as possible evidence; when it is done poorly, it is trumpeted as absolute proof of someone's contention.
How flawed is the system? According to @Kosmonaut (replying to a different meta question),
[T]here is a ton of metadata error in Google Books — enough to possibly be concerned about a lot of the conclusions one might draw from it. As it happens, I am working on a project that was originally going to use some Google Books data, but in-depth analysis seems to indicate that dates are way off (as in, 25% of the pre-1800 tokens I have looked at so far seem to be off on their publication dates by an average of ~100 years).
There is also the matter of comparison terms. Sometimes one can find comparisons that will work, but very often the terms can't be compared exactly. For example, in this answer one of our high-rep users attempts to adduce an NGram to answer a question about nuances of meaning:
One wonders how relative frequency is germane to any discussion of the meanings of those words.
Another flaw involves book results being used to draw inferences about spoken language as well, or at least to conflate the two. In this answer [comment chain since then has been deleted for unclear reasons], another contributor uses NGrams to show that one usage is vastly more common than another, which obviously feels counter-intuitive to him because he admits in the comment that "FWIW, it surprised me too. I would probably say 'go for a swim' myself." The conclusion I draw is that Google NGrams are a hammer looking for a nail.
Here is another case where someone draws a faulty inference based on an NGram search:
I leave it to the eloquent @MrHen to debunk the chart:
How does that NGram support any particular usage? Isn't it just tracking uses of "a week hence"? How would it know if that hence was for the future or not?
Even when they can define their terms, users of NGrams succumb to the simpleton awe inspired by the sight of a dramatic projection of minuscule data points to show great trends and differences. They ignore the scale on the Y-axis, which reports the differences in a range that may be only a few cases in tens of millions.
And very often NGram answers garner upvotes because people don't think about the data behind the charts and simply look at the pretty line graphs and upvote the answer.
I'm not against Google NGrams. Used intelligently, they can offer some valuable information. But I believe them not to be the unimpeachable source people pretend they are on EL&U. I'm asking for some kind of standards. I don't know what kind, but I hope you all have some suggestions.
As if to prove my point, we have another NGram travesty "proving" a point with faulty data. Because of a bug (or quirk, if you will) in the Google NGram Viewer, hyphenated words invariably flatline. A hyphenated word needs to be separated into a trigram by adding spaces around the hyphen. Notice the difference between these two charts:
Sorry I switched the colors. But at least I resurrected "upper-case" from oblivion. But there is another flaw in this chart — can you spot it? How about cases where "upper case" is not used as an adjective describing a letter? Or how about cases where, even though the subject is typography, one is trying to use case as a noun: "He used the upper case quite a bit in his emails." Of course, the author's point about usage is probably in the main supportable, even without reference to these charts. But given that these charts were introduced as evidence, the answer is not really accurate, and accuracy is something we should at least strive for on this site. We aren't always 100% accurate — nobody's perfect — but if we passively give the nod to glaring errors, relying instead on the base feeling that whatever garners the most votes must be true, we become little better than the Urban Dictionary.
Finally, for your amusement and to show the relative scale of usage we're talking about, consider the following series of charts. Note especially how one single high-frequency word causes all the rest to flatline. And none of the other words may be considered obscure in the least! (Also, it is obvious from an NGram search that apples are better than oranges. Or that people like them more. Or that ... well, draw your own conclusions. The data are there for your interpretation.)
Use of "is" in the corpus peaks at 1% — one out of a hundred words. Use of those other common words is negligible by comparison. And use of obscure word combinations may not even be worth talking about from a genuine statistical point of view. If there is a statistician among you who can define the terms for such a discussion, I am open to being educated. Until then, I will look at answers involving NGrams with a jaundiced eye.
See this question and its ham-fisted NGrams, some of which have already been deleted, for more evidence of NGrams Gone Wild. At one point, two answers used the same query term and arrived at opposite conclusions. You can't make this stuff up, folks.
Even I have not been immune to the lure of Google NGrams in the past. @Mechanicalsnail pointed out to me an instance of my own seduction, which was written before I had fully grokked how NGrams should and should not be used.
AND NOW THIS, from PLOS|ONE:
Overall, our findings call into question the vast majority of existing claims drawn from the Google Books corpus, and point to the need to fully characterize the dynamics of the corpus before using these data sets to draw broad conclusions about cultural and linguistic evolution.
A tip of the hat to MattEllen.