I was wondering if anyone is aware of a similar service to Google Ngram for words used on the web.

To clarify, I'm not looking for the frequency of search terms used on the web (i.e.Google Trends) but the actual frequency of words used across many websites over time.

migrated from english.stackexchange.com Oct 24 '13 at 12:49

This question came from our site for linguists, etymologists, and serious English language enthusiasts.

  • I'm not sure this question fits the criteria of this site. However, I'm not sure where to post it. If anyone can point me in the right direction that would be much appreciated. Alternatively if they could confirm this is the correct site to use. That would be great. – jamesmstone Oct 23 '13 at 9:11
  • I think this is a fine and interesting question to ask, but ultimately unanswerable. Any such service would be terribly incomplete. For starters, "the web" is not clearly defined and in fact escapes definition, and even if we agreed on a definition of "such and such precise subset of the Surface Web", it might still be unanswerable for technical reasons. You'd have to crawl it all, filter out the typos (or not?), etc. Lastly, millions of words are added to the Web this very moment (and I'm helping out), so any stats would be obsolete before they even got compiled. – RegDwigнt Oct 23 '13 at 9:29
  • But of course if, say, archive.org had some stats for various time stamps, it'd be fun to look at them. – RegDwigнt Oct 23 '13 at 9:31
  • @RegDwigнt yeah I guess it would be unpractical. However, I do like your archive.org idea. After reading your response I was thinking even if a services like google recorded the frequency of each word across its index of the web as it scans sites for changes. It would be interesting to see "break out" words like with google trends. – jamesmstone Oct 23 '13 at 9:38
  • Yours is a very specific request. FWIW, there are free word frequency lists on the web, such as the Word Frequency List of American English, although that's not exactly what you wanted. I suggest you use the contact info on www.wordfrequency.info. – Talia Ford Oct 23 '13 at 10:33
  • There is a corpus of web-based English available. It's called GloWbE, the Global Corpus of Web-Based English. However, I don't think it will give you information with respect to time. – snailboat Oct 23 '13 at 14:52
  • This would be a fantastic question for [meta]. – MrHen Oct 23 '13 at 14:58
  • As @MrHen says. Suggesting migration to meta. – Kris Oct 24 '13 at 12:49

You must log in to answer this question.

Browse other questions tagged .