0

I'm trying to build a schematic table/diagram with all the possible combinations of:

  • verb tenses (Past Simple to Conditional 3)
  • form (Affirmative to Interrogative Negative)
  • auxiliary verb conjugation (was/were, etc)
  • word order (like in Interrogative Negative where to meaning can change)

Also I'm trying to get an example phrase from movie quotes for each and every possible case.

It's been quite a long journey, maybe because I'm a perfectionist and I want to make it the most symmetrical/organised possible (to be very straight-forward and easy for one to understand it), and as you can assume I couldn't use Chat GPT for it because of its hallucinations.

Hence my question: "Is there any NL model reliable enough to do the hard work for me?"

6
  • 2
    Are you talking about transformations? Here's a list to start with. Aug 20, 2023 at 22:27
  • Hi @JohnLawler. As I'm not from the field of Linguistics, but only a curious data scientist, I'm not able to dive into the depths of the English grammar theory. I'm rather looking for something more pragmatic, like a tool really, as chat GPT, but reliable enough so I could just build my diagram of verb tenses.
    – MD11
    Aug 20, 2023 at 23:05
  • 1
    I can't help but think that AI is unnecessary for this. Aren't there lists of these things online as resources for learners? And once you have your lists ready, you can use the "Movies" corpus to find quotes. (Also this isn't really the right place for questions about AI, but I'm not sure which site this would fit on.)
    – Laurel Mod
    Aug 20, 2023 at 23:34
  • 2
    The so-called third conditional is not a tense but a "mood" I don't know how useful that is to you but some people are very precious on the naming of things. Also, the 3rd conditional is a misnomer in the sense that there are dozens of ways to form a conditional, so I'd be wary of any programme that coerced any language to appear coherent, logical and dissectible.
    – Mari-Lou A
    Aug 21, 2023 at 8:20
  • 1
    So you want something like ChatGPT but 100% reliable? I think if the engineers knew how to make ChatGPT 100% reliable they'd do that. You're chasing a chimera, and as others have said, need to find a better way of doing it. (Questions about why this is impossible belong elsewhere.)
    – Stuart F
    Aug 21, 2023 at 14:10
  • ChatGPT output uses good printed grammar for the most part. It's much more reliable than newspapers or undergraduates. You can't rely on what the output means because it doesn't mean anything; but the grammar is generally OK, and you're likely to get good examples of virtually every possible syntactic construction that's been written if you sort through enough output. Aug 21, 2023 at 16:04

1 Answer 1

1

TLDR You could use an LLM to help you but don't rely on it to do the whole thing correctly for you in a single prompt (request).


I would find it unlikely for an LLM (like ChatGPT or LLaMA) to produce systematic factual information like that either now or in the future. There might eventually be processes that use an LLM as one component that could do it.

The reasoning here is that an LLM gets a statistical picture of all its training data (books and websites and such) - they do not operate on facts or abstractions of patterns by meaning. They tend to reproduce text that looks factual or like an abstraction because the most statistically likely patterns will involve the words that come in a sequence that happen to be facts (or abstractions).

For example, 'This banana is ...' and 'Cars are a kind of ...' will usually get 'yellow' or 'vehicle' respectively to follow, not because those exact sentences have been seen in the training corpus before, but because those words and nearby words with similar cooccurrences are more likely. Another way of saying this is that LLMs are in some sense hallucinating -all- the time, we just don't notice when reading outputs unless the random sequence is wrong factually.

It could be that someone on the internet has created exactly such a table, or at least parts of it. LLMs are not web searches (the training corpus is a fixed subset of the internet - it's not up to date and it is not all of the internet (which is not even all of human knowledge). But an LLM can sort of extract a lot of patterns out of that data that is sort of like Wikipedia.

That said, an LLM can probably get you a lot of that work for you. It may not be complete, it may have a few errors, and it may give you more than is intellectually supportable. If you already know a lot about the subject that's in your table, then I think an LLM would be a good tool for you to help you start off and compile examples and to format the table. But you should use your expertise to manage all that.

If you're not an expert in the subject, then don't rely on anything that is factual. It can give you a good rough idea but any individual 'fact' should not be relied on. (kind of just like wikipedia).

I'd suggest using an LLM incrementally but also to revise and go back a step.

For your exact subject matter, you probably want to ask first what the main categories of verb forms are explicitly saying 'For example: aspect, mood, tense, ...' Or 'statement, question, negation...', things you already know to be the case. Then refine and ask about the different tenses, the different moods, etc.

At some point you can then prompt your LLM to 'combine all these into one table...'. It will most likely not be exactly what you want but then you can repeat pieces of this process to improve it little by little.

I think you'd be able to produce a table that is good enough for you, but I'm not sure it would be of a quality that would be universally acceptable.

(you definitely can do this same process for other languages, but you probably want to do it -in- that other language and have different categories as starters)

5
  • I tend to agree with you in " LLMs are in some sense hallucinating -all- the time...".
    – MD11
    Aug 27, 2023 at 4:12
  • 1
    And also I think I'll just give up trying to find a simple magical solution and continue doing using ChatGPT for searching phrases case by case.
    – MD11
    Aug 27, 2023 at 4:14
  • But thanks for the help man.
    – MD11
    Aug 27, 2023 at 4:14
  • @MD11 yes case by case (and check each output for accuracy (ie don't expect it to always be right), but also use it at higher levels of abstraction eg for the different categories. You'll still have to do work, but it will be a lot less than doing everything from scratch.
    – Mitch
    Aug 27, 2023 at 14:29
  • The point is, the output may be wrong -- good chance, in fact -- but the language it delivers the output in is grammatical, idiomatic, clear, and safe to imitate with regard to tenses. Any LLM output uses tenses correctly, in sentences, paragraphs, and longer texts. So you can always get a model to copy. That's a useful trait. Oct 2, 2023 at 19:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .