3

Where can I find a large corpus of contemporary English writing?

I am a computer programmer, and I would like to execute a program which searches the corpus for certain text-patterns.

I am willing to restrict my search to books.
We can ignore news articles, essays etc...

Actually, a corpus of book titles is sufficient.

If I had a list of 10,000 titles and authors, I could easily obtain digital copies of the books themselves.

I am open to alternative suggestions, but maybe I could use a list of the titles of all books which have ever appeared on the New York times best seller list.

After obtaining a list of NewYork times best-sellers, I can look-up the year of publication for the first editions of each book.

Our goal is to obtain a list of contemporary works.

I would like to remove Moby Dick (1851), etc... from the list.

As such, I can eliminate all books whose first edition was published before the year 1960.

Both fiction and non-fiction are acceptable.

Where can I obtain a list of the titles of all books which have appeared on the New York times best seller list?

The New York Times website mostly has only reports weekly and monthly data, not cumulative lists.

11
  • The NYT itself is a pretty large corpus of contemporary English.
    – Lawrence
    Jan 13, 2021 at 16:40
  • I have voted to move this to meta, but it may be a dupe over there.
    – Cascabel_StandWithUkraine_
    Jan 13, 2021 at 16:47
  • You can also look at Twitter, Quora, and Wikipedia for samples of contemporary English writing.
    – user409914
    Jan 13, 2021 at 17:11
  • 3
    We have a list of English corpora in the list of reference works.
    – choster
    Jan 13, 2021 at 19:07
  • In a library???
    – Hot Licks
    Jan 13, 2021 at 20:13
  • 1
    I’m voting to close this question because the number of obvious sources is huge and the OP shows no signs of having done any research.
    – Greybeard
    Jan 13, 2021 at 20:36
  • @HotLicks most libraries would be unacceptable, because I require all writing samples in the corpus to be digital computer files. Books made of paper and ink are completely unacceptable.
    – Samuel Muldoon
    Jan 14, 2021 at 20:00
  • Cascabel. The META site is the wrong place for this question. Suppose people post recipes on recipe.com. Then, meta.recipes.com is a website for discussing the website recipe.com. Almost nothing about cooking or baking should appear on meta.recipes.com. Instead, users ask, "can we change the website color scheme from blue and gold to yellow and brown?" Also, "add we add a button which automatically converts imperial units in a recipe to metric units?" meta.stackechange.com is for questions about the website itself.
    – Samuel Muldoon
    Jan 14, 2021 at 20:13
  • Whoever moved my question to meta... my question is NOT about stackexchange.com. I asked where to find a digital library of copies of contemporary English writing. That has nothing to do with the stackexchange website itself. You don't understand what meta is for. The meta site is for questions about the website. For example, "How do I post a question?" or "I do not like the color scheme of stackexchange.com, can we change it?" or "How many downvotes is required before a question gets deleted?"
    – Samuel Muldoon
    Jan 14, 2021 at 20:18
  • Samuel Muldoon, this is not meta.stackexchange.com. This is english.meta.stackexchange.com. General requests for resources (yours is one such) are appropriate here, but are not appropriate on ELU main.
    – JEL
    Jan 15, 2021 at 5:57
  • I voted to close because the question already has an answer at What good reference works on English are available?.
    – JEL
    Jan 15, 2021 at 9:00

3 Answers 3

Reset to default
2

The Corpus of Contemporary American English (COCA) is exactly that, and it can be downloaded in its entirety. (On the landing page, there is a place that says 'Download the corpus (and corpus-based frequency data) for offline use'.)

The same website hosts several other corpora of English, e.g. the British National Corpus (BNC), the TIME Magazine Corpus (here), etc.

1

From https://gutenberg.org

Website Terms of Use
Audience

This website is intended for human users only. Any perceived use of automated tools to access this website will result in a temporary or permanent block of your IP address.

  • If you want to download many books (i.e., more than ~100 per day) manually or using an automated download software, download them from one of our mirrors, not from the main site. See the list of PG mirrors and the roboting guidelines.
  • If you want a list of all our books, download and save the Gutenberg index file It can be opened with any browser or word processor.
  • If you want a machine-readable database of all our books, read the Offline Catalogs and Feeds page.
2
  • In my question, I specifically stated that all English writing should be published after 1960. The vast majority of books uploaded to project gutenberg.org are very OLD. For example Charles Dickens, Jane Austin, Lewis Carol, Johnathan Swift, Charlotte Bronte, Bram Stoker, Oscar Wilde. Most people fluent in contemporary English could not read Moby Dick if they tried.
    – Samuel Muldoon
    Jan 14, 2021 at 20:09
  • I beg to differ, but about the are you are likely correct
    – mplungjan
    Jan 14, 2021 at 20:34
-1

There is already Google Ngram Viewer that does what you propose.

6
  • 1
    "Where can I obtain a list of the titles of all books which have appeared on the New York times best seller list?" - NGram viewer gives a list of titles it uses? Do you have a more specific link?
    – Mitch
    Jan 13, 2021 at 20:00
  • @Mitch I suspect that the OP threw in the NYT as an example, however, you would get it from the NYT or a Google search that took me 30 seconds: westportlibrary.libguides.com/NYTimesbestsellers. The sources are the least of the OP's worries - copyright will come into it and the whole is limited by the number of titles in electronic form and the access to each type of file. I will be voting to close this question as showing insufficient research.
    – Greybeard
    Jan 13, 2021 at 20:34
  • Can you update your answer with a link to the list of sources that NGrams has?
    – Mitch
    Jan 13, 2021 at 22:25
  • @Mitch I think you may be missing the point the question, which is "Where can I find a large corpus of contemporary English writing?" (Google Books) The reason for this is "I am a computer programmer, and I would like to execute a program which searches the corpus for certain text-patterns" which is exactly what Google Ngrams does - it can even disregard books published before 1960!
    – Greybeard
    Jan 14, 2021 at 0:54
  • The first line asks for a large corpus. But then elaborates and specifies that they want to do the searching for certain patterns (unspecified) -and- control for which work it came from. I don't think Google NGrams has that... or maybe it does...it'd be nice if they made that available. If you elaborated your answer to include the patterns searchable -and- how to restrict the search according to work, that would be a great answer.
    – Mitch
    Jan 14, 2021 at 2:29
  • @Mitch The idea of giving instructions on how to operate Google Ngrams to advantage and/or at an advanced level is not required - it would be wrong to do so. Google Ngrams already has pages of explanation upon which it would be difficult to improve, and pointless to summarise. The OP is reinventing a wheel. They need to look at Google Ngrams themselves, see how it operates, and understand it. They can then ask themselves where, if anywhere, their project needs to go.
    – Greybeard
    Jan 14, 2021 at 10:58

Not the answer you're looking for? Browse other questions tagged .