I'm a NLP researcher and am looking for a English dictionary dataset to train a language model? Any suggestion? The Oxford English Dictionary (OED) right meets my need, but it seems that they rarely reply to anyone's demands. So is there any subsitution? At least containing words, senses and corresponding quotations(more than 2 is better). Thanks a lot!

  • 3
    Programming resources like this are't usual here. I see questions like this with answers over at [se.se] often enough. 1) OED doesn't have a public version and is not a programming resource. 2) WordNet is the classic word meaning ontology (words and related senses) that you can download for what it looks like you want, but it doesn't have usage examples. 3) You probably don't want to train from scratch - use spacy. If you ask elsewhere, make sure you give details on what you're trying to do, what you expect your training to accomplish, what the inputs and putputs of training are.
    – Mitch
    Aug 9 at 15:24
  • See also: english.meta.stackexchange.com/questions/3591/… which is sparse and out-of-date. It should be updated to say 'Just use Python and spacy'
    – Mitch
    Aug 9 at 17:00
  • So basically I need quotations alined with each word meanings
    – Charles
    Aug 10 at 2:55
  • 2
    Charles, dictionaries are not actually defining unequivocally the meaning of a word, even with a quote. They are just hints to humans for things they already know but have.a hard time articulating. A single quote for a meaning may give just enough to a human to 'get it', but since it has no context a machine will do poorly on word semantics using a dictionary. You may want to look up the word2vec method which assigns a vector to each word which approximates some of the meaning. The vector is established by millions of sentences ('quotes') but is still an approximation to word meaning.
    – Mitch
    Aug 10 at 12:30
  • I'm partial to the Cambridge Dictionary for referencing here in the site. It is not behind a paywall like OED, offers multiple examples and includes variations (formal, business, US, UK, India) Sep 16 at 13:11

You must log in to answer this question.

Browse other questions tagged .