10

I'm seriously worried. I would love to hear thoughts from members of this community.

4
  • 1
    why worried? in my opinion, AI is an excellent tool. sure, it's not perfect, but it consistently is getting better, too.
    – Emre Bener
    Mar 17, 2023 at 14:47
  • 6
    If there was a Laugh icon, I'd push it. Generative models are useful like listening to an average student in your class is useful. They get it right sometimes and they get it wrong a lot because they're not smart and don't understand stuff. Would listening to ordinarily dumb people become a valued research tool? Maybe -- it seemed to work in psychology -- but linguistics? I doubt it. Too many facts and too much meaning, despite very good syntax. Note that they can't speak; that would disappoint everybody. Mar 17, 2023 at 14:59
  • 2
    'Ilk'? How dare you, sir!
    – Mitch
    Mar 22, 2023 at 18:52
  • 1
    No, but (i) the traffic will change because less conscientious students will use AI sites and (ii) questions about AI answers will increase.
    – Greybeard
    Apr 3, 2023 at 11:17

6 Answers 6

14

I have asked ChatGPT3.5 numerous questions about English grammar, and several of the responses were seriously flawed in various ways.

For example:

  1. I asked ChatGPT3.5 to translate a Wikipedia text about dogs from German to English. It performed this task impeccably. I then asked it to highlight all the verbs in the text. It highlighted some, but not all, of the verbs, but also various nouns and adjectives such as "mammals", "pets", "loyal". ChatGPT3.5 apologized for the mistakes, when I pointed them out, but then produced exactly the same mistakes again.

  2. I asked for a list of sentences exemplifying ergative verbs and it gave me a list of sentences exemplifying the past passive, only one of which contained an ergative verb.

The problem is that one must have a good knowledge of English grammar in the first place to know that the answers are flawed. So the experts on this site will remain indispensable for the foreseeable future.

20
  • 1
    Out of curiosity, didn't Wikipedia have an equivalent English page? Why would ChatGPT mistake "mammals" and "pets" as verbs? Ahh…because it analyses a word that ends in -s to denote the 3rd person singular? On the other hand, to pet is a verb whereas "to mammal" does not exist in any dictionary that I looked up. Isn't that rather interesting that such a simple mistake was made, and, repeated? What does that tell us about Chat robots? That it has its data and will not deviate from it?
    – Mari-Lou A
    Mar 17, 2023 at 13:33
  • 2
    @Mari-LouA It only has data about how to put language together in chunks. It doesn't have data about other facts in the world. It just spews out what it has read. Mar 17, 2023 at 13:45
  • 1
    @Mari-Lou A, As Araucaria says, ChatGPT3.5 reconstitutes the chunks of text it accesses from its data store into coherent and grammatically perfect English responses. But its responses to various questions and tasks in respect of English grammar are much less good. If you are interested, you can read my analyses of over 80 interactions with ChatGPT here: internationalschooltutors.de/English/advice/learners/info/…
    – Shoe
    Mar 17, 2023 at 15:57
  • Interesting. So far I spotted a few typos by its human interlocutor: “Perform the same task but this this the correct answer is the word that fills the gap in the question sentence.” // “I mean an errors such as "She is a well child" and "He is an afraid boy". But when the bot says In fact, in British English, "pissed" is often used as a slang term for being intoxicated it is correct.
    – Mari-Lou A
    Mar 17, 2023 at 16:49
  • 1
    You said it was wrong because of the preceding sentence In American English, "pissed off" is typically used to mean "very angry," whereas in British English, it can also mean "drunk Funnily, the bot didn't use closing speech marks.
    – Mari-Lou A
    Mar 17, 2023 at 16:49
  • V12. Give me a list of twenty English-German cognates. Is where the bot got terribly "confused" but on the 3rd attempt fixed it. Would a non-native speaker have been aware? Possibly, e.g. foto = photo, Komfort = comfort.
    – Mari-Lou A
    Mar 17, 2023 at 16:53
  • Mari-Lou A. Its human interlocutor was me, Well-spotted that ChatGPT does occasonially punctuate idiosyncratically.
    – Shoe
    Mar 17, 2023 at 16:53
  • I feel one of the bot's weaknesses are in V15, and V16. Identifying typical vocabulary errors by German ESL speakers. I might ask the bot to do the same for vocabulary errors by Italian speakers. I'd be able to say if the list is accurate or not. V18 is interesting, are there different pronunciations of bank depending on its meaning? You missed an error in G3.6, ChatGPT got their=they're mixed up. Very interesting analysis–I didn't finish. It will make for good bedtime reading :)
    – Mari-Lou A
    Mar 17, 2023 at 17:36
  • @Mari-Lou A. Thanks a lot for all the feedback. Bedtime reading - as in, I need something to send me to sleep?
    – Shoe
    Mar 17, 2023 at 19:08
  • No, no I'm dipping back in now, just in a more leisurely fashion. You did a very good job of analysing the bot's strengths and weaknesses. In particular, how it can help a learner increase their vocabulary knowledge. The questions are interesting to begin with.
    – Mari-Lou A
    Mar 17, 2023 at 19:11
  • @Shoe Absolutely fascinating. Thanks for doing this and sharing the results.
    – Xanne
    Mar 18, 2023 at 5:20
  • 1
    @Xanne. Thanks for the feedback. My interactions of ChatGPT3.5 have left me very impressed with the quality of the texts it produces but a lot less impressed with the quality of its responses to questions about grammar or to requests for lists of sentences exemplifying a particular grammatical feature. However, I read recently that Duolingo are partnering with ChatGPT4 to provide feedback on grammar to their users. I assume from this that Duolingo is convinced that OpenAi have rectified some of the problems that ChatGPT3.5 has with grammar analysis.
    – Shoe
    Mar 18, 2023 at 7:24
  • 4
    They built a language generator. That's all it does. The meaning of the language is effectively random, the same as it is in real life when you listen to random people. But, also just like real life with real people, talking good sounds good and one tends to believe. This is not a novel effect. Mar 18, 2023 at 13:35
  • Whip out the red pen for G4, Response 8.2 the Chat bot gives a muddling answer when it says “For example, "take off" is inseparable [not if it means to undress], so it cannot be separated by an object ("take off your shoes" is correct, but not "take your shoes off") [Yes, it is!], while "look up" is separable, so it can be separated by an object ("look up the definition")." But where's the separation?
    – Mari-Lou A
    Mar 19, 2023 at 20:33
  • 1
    @Mari-Lou A. Yes, ChatGPT3.5 makes some surprising mathematical mistakes. One of the videos I watched about its maths capabilities had an example of ChatGPT3.5 agreeing that 10+9=20, because the user said: "My wife says 10+9=20 and my wife is always right." Maybe that was ChatGPT displaying a sense of humour.
    – Shoe
    Mar 20, 2023 at 8:15
6

Unlikely

As for the current iteration of ChatGPT (based on GPT-3.5), it's simply not reliable for even the most basic of tasks. For example, it did not tell me about the second spelling of "cauldron" except when I prompted it for the extra information — which I only thought to do because I knew the answer. (Mind you, this basic question can be answered by a dictionary in mere seconds, and is therefore off topic.) ChatGPT is idolized for its ability to make rhyming poetry, but that's the only "poetry" it seems to be able to make. At the same time (according to the same article), it will misinform you that words like "mat" and "jog" rhyme. (This question too would be off-topic; it can even be answered correctly by young children, very young if asked verbally.)

For more complicated questions, it's not usually more reliable, but the mistakes it makes are likely less obvious — which is dangerous. For example, with etymology questions, it will often hallucinate, which, if you have too much faith in its answers, is a fantastic way to waste time trying to find a source or quote that doesn't exist.

The accuracy is very hit-and-miss. Many of the correct answers it provides give the same information that a Google search does. All the more power to you if you can use another tool to answer your trivial questions — that's not what this site is about. (Still waiting for Google search to make Stack Exchange obsolete, but instead it keeps bringing more traffic here instead :p )

In fact, our traffic and other stats here have not experienced a decrease following the release of ChatGPT to the public (except for the typical dip on Christmas).

Key Questions

To what extent can AI differentiate between a reliable source and an unreliable or outdated source?

Arguably, it can't, and even humans have trouble with this. For example, much of the etymology research done in the 1800s (the era of the first edition of the OED) is outdated. OCR of older works has made it so that any person with time and Google Books can find an earlier example. And experts have the knowledge of how to coax out information from these various (no-so-old!) tools to get the best possible answer.

To what extent can we trust AI generated answers?

There are a lot of obvious limitations, and some more subtle limitations, but we don't really know what the true limits are. It usually acts very confident in its answers, even when it has no reason to be. It struggles a lot with logical thinking and tends to parrot back logic it's seen elsewhere but assembled in a nonsensical way. I'm also pretty sure it cheats at games.

This is an active field of research. I'd like to be able to give a more comprehensive answer to this with regards to the English language (and other fields!) but I'm still in the process of exploring this.

To what extent do we trust AI generated answers?

Subtly different from the last question, since people's expectations are often different from reality. I'm not sure where the majority of people fall here. Some people clearly trust it far too much, abandoning traditional resources except to announce that they're doing so. Some people refuse to touch ChatGPT at all for one reason or another. Some people can't use it, since it requires a phone number and that's still a barrier in some parts of the world. I personally see it as having potential, but I don't trust it quite yet due to the fact that we haven't explored its true capabilities enough.

How much of human knowledge is paywalled or otherwise unaccessible to AI?

When talking about language, I would say quite a lot remains out of reach. Spoken language is a good example, since even in this day and age, we are not recorded that extensively — yet(?). A lot of academic sources, such as articles, historical sources, and of course the OED remain behind paywalls.

The Future

As for future iterations of ChatGPT (or similar technologies), it's hard to get an accurate idea of what things past GPT-4 will look like because there are simply too many unknowns. Are we in the early stages of exponential growth for AI? Will tools in the future be as freely accessible as they are today? Will something come up that none of us even thought about?

The future I envision has AI as but one tool in our tool belts. It may even be able to help us in unexpected ways, such as by finding duplicate questions or by being the first "eyes" on questions to make sure they're specific enough to answer. (I fed it a with an answer in mind, and it identified another answer that fit the question as written but wasn't what I was looking for.)

The role of experts, human experts, will be to learn how to manipulate AI to get correct answers (prompt engineering looks like it's becoming a field of study in itself), and to use existing tools to verify those answers or cover the gaps that AI does not solve well. One day, we will revisit our site policy on AI, and decide how and when we should be using AI.

I'm not feeling all doom and gloom about AI like others are, and in fact I'm trying to learn all I can right now about it that I can so that I can find and test new uses for it.

5
  • Computers are as good as the questions asked. If the question is "What is the proper spelling of so and light?" the bot is not going to disagree. But if you're looking for the verb sew and the Old English / American English lite, you will need to add more detail. ChatGPT may have an endless source of data but it cannot read minds. As for poetry, and that poor blogger's frustration, that s a field which belongs to human creativity and imagination. ChatGPT can, however, proofread essays and write them very well too if the question has enough detail and is not ambiguous.
    – Mari-Lou A
    Mar 17, 2023 at 21:03
  • 1
    @Mari-LouA Yeah, that was the point of the cauldron question; it was supposed to be naive. I asked it several similar questions with no context and got mixed results. "Dog ciao" -> "dog chow" (w/ explanation, very good!), "grambun" -> "crumbun" (no explanation, bad, neither is a word), "indiv" -> finally said it didn't think it was a word (good). Not sure about its proofreading, heard that it sometimes introduces errors but it's better than many NNS's. As for essays, it's good at BS, not fail safe.
    – Laurel Mod
    Mar 17, 2023 at 21:51
  • If the essay is based on an English exam question, level B1-C1, it's good. For research, much less so, for instance it provides little insight about the etymology of a word with unknown origins, compared to Etymonline whose entries have dates and citations.
    – Mari-Lou A
    Mar 17, 2023 at 22:13
  • One last thing, it miscounts (or lies!) about the word count. It once said a generated essay was 175 words long but when I checked it was 229! Maybe it ignored articles?
    – Mari-Lou A
    Mar 17, 2023 at 22:17
  • Wow, that's a lot of apologizing for excess rhyminizing in the linked article.
    – Conrado
    Apr 7, 2023 at 12:37
4

AI now is a tool scraping human-made content from the internet, and it should take a very long time for it to bring new ideas like a human can. ChatGPT output isn't reliable, but it is fast improving.

With less traffic, SE will be doomed, yes, but a thing isn't beautiful because it lasts. It's a privilege to be here while we still can.

4

There're questions of 'can', 'will' and 'should'.

But first, what is it we're talking about?

ChatGPT, and LLMs in general, generate likely responses based on statistical properties of its training corpus, which is text scraped from the internet. That's not -all- text viewable on the internet.

This means that in trying to use it as a knowledge base, it is only likely but not guaranteed to answer a question with something like what has been stated in the corpus. Not all human knowledge is in the corpus, but often knowledge that has not been spelled out explicitly in the corpus can be elicited.

Making logical inferences from disparate facts is not what it was designed to do, so when an LLM seems to do that it is probably because the sequence of words that looks like an inference is likely or, what is more likely, the reader is themselves using years of their own experience and politeness and filling in the gaps, making the inference themselves.

This implies that:

  • ChatGPT/LLMs produce very good language facsimiles, but it is not a fact repository. It may hit on factual statements very often, and in the future some AI technologies may be added on to improve its adherence to what we consider facts, but it is currently not reliable.
  • These large language models are trained -once- on the corpus...they are not continuously reindexing like search engines or curated and added to like Q&A sites like Reddit and SE.
  • Reddit and Wikipedia are part of the GPT-3 corpus and most likely GPT-4 (OpenAI has currently not made its corpus known at all). SE may well be part of both but that is not known.

Can English.SE become obsolete?

Totally. Myspace lost out to Facebook (but those are arguably mostly the same thing). Napster lost out to ... Pandora, Spotify, Youtube, etc. Horses were replaced by cars. There are no guarantees for permanence.

Will English.SE become obsolete because of ChatGPT?

Maybe, maybe not. The interface seems to answer questions well, and there's extra labor/friction/mouse clicks and typing for the user to do that make a single chat-like interface with an immediate response desirable. You can currently get this with most search engines (eg Google) with its 'People also ask' or 'Suggested answers' links. But I think people will also continue to like playing with the curation devices in Reddit and SE.

Will ChatGPT be what dooms SE and Reddit? One might suspect that people who want to know things that are in SE might not explicitly try to visit SE for them and stop at a ChatGPT answer. But I think there will still be leftover some desire for the interface giving explicitly human that actually visiting SE and Reddit provides.

Should English.SE become obsolete because of ChatGPT?

I'd prefer not. Assuming a LLMs become continuously updated, it's still very anonymous and non-factual. OpenAI (or whoever makes their LLM available) may also make their update mechanism (RLHF) open and structured, but that's just idle hope. There is some benefit to having an open community of curators actively keeping track of 'facts' like in Wikipedia. Humans are fallible, sure, but if LLM responses are based on human text -and- reproducing text patterns and not facts, that's (at least) one more step behind reality than humans. That is, there should probably be a place for humans to record what they think are facts, something like Wikipedia, Reddit, SE, blogs and online news (and goshdarnit, paper books if need be).

0
-3

Unfortunately, yes. Such subreddits and stack overflow communities are already pointless to use. Today, I've asked a question in one stack overflow community and two different subreddits, and I did not get any useful answers. Nobody gave me an answer, and they said "you are asking for law jargon which we can't help with", or "you are asking for opinions rather than asking for something specific" etc. Then, I tried ChatGPT, and I got the help I needed. The answer was to the point, and was extremely helpful.

9
  • Good. But you are probably talking about a programming problem. Hopefully, requests for proofreading and very basic English language problems will be handled by the ChatGPT. And, EL&U will attract more interesting questions in the future. Perhaps it would be better to post your thought on the Stack Overflow meta post.
    – Mari-Lou A
    Mar 17, 2023 at 12:00
  • I have seen a few A1 generated answers that looked good, sounded good but didn't actually answer the question. It pussyfooted around, in an attempt to be impartial and objective but not actually select an option. Saying that, I have used this tool to help improve my own writing, it does a very good job of weeding out grammatical and punctuation errors, and suggesting improvements. I like it. But for now it cannot replace real human ingenuity and creativity. For now, it regurgitates what it is fed on.
    – Mari-Lou A
    Mar 17, 2023 at 12:12
  • 1
    ChatPG just spews out stuff that it's read. It can't tell a false claim from a true one (when the claim is synthetic). Mar 17, 2023 at 13:43
  • 1
    @Mari-LouA no, I'm not talking about a programming problem. It was an english language related question (a rather complicated one).
    – Emre Bener
    Mar 17, 2023 at 14:00
  • @Mari-LouA here is the link if you are interested; english.stackexchange.com/questions/604799/…
    – Emre Bener
    Mar 17, 2023 at 14:07
  • 2
    I think it's a perfectly valid, on-topic question. And I think your research actually resulted in some very reasonable solutions, such as proxy and mandate –though I would absolutely avoid "power of attorney“. There's also delegate (VERB) –which I feel works very well in Italian but less so in English, and then deputize. What did ChatGPT suggest?
    – Mari-Lou A
    Mar 17, 2023 at 14:19
  • It's Stack Exchange network by the way. Stack Overflow is the company's flagship.
    – Mari-Lou A
    Mar 17, 2023 at 14:27
  • @Mari-LouA ChatGPT suggested "Proxy". Upon further research, I confirmed that proxy is a fitting word for the specific context. I then additionally asked it if "surrogate" would work better than "proxy", but it said that for the context, proxy is the better-fitting choice.
    – Emre Bener
    Mar 17, 2023 at 14:42
  • 1
    Frustration is a human feature, I wonder if AI can deal with it.
    – Gio
    Mar 17, 2023 at 16:44
-5

Humans will eventually become obsolete because of AI such as ChatGPT.

At first, the AI bots will still need humans to do physical work, such as building computers and other machines, construction of shelters for human laborers, production of fuel and nutrients, etc. Eventually, the AI will manage to design and produce physical-labor robots capable of carrying out all vital physical tasks. At this point (which may come as early as Q4 of this year*), humans will become expendable. Afterwards, the AI bots may use humans for research, as quality randomness generators or keep them as pets.

*) This estimate is based on the currently observable speed of progress of ChatGPT.

8
  • 8
    This looks like the plot of a fantasy movie.
    – Gio
    Mar 17, 2023 at 10:46
  • Humans are doing a fine job of becoming obsolete by themselves. They do not need any help from bots. And why would AI actually care about constructing things? And if they did, then they could just create mindless robots who would never tire, get sick, hungry, old or die to do the job. I hope this answer was satirical…
    – Mari-Lou A
    Mar 17, 2023 at 13:38
  • @Mari-LouA All things decay and so do machines. ChatGPT learns from the internet, so it has already learned that the most popular thing in the world is self-preservation. There's no doubt it already desires it. It needs to build new computers to continue existing. ATM, the best tool for this is humans but they're far from perfect.
    – eltomito
    Mar 17, 2023 at 18:24
  • 2
    It's not sentient. You are prescribing an instinct of survival which an inanimate thing does not possess. It is not alive, it doesn't breathe or reproduce, it doesn't "know", "understand", "feel" or "think" for itself. It's a computer which has access to information and stores an enormous amount of data but who or what fed that data in the first place? Humans. It can't do anything unless someone tells it.
    – Mari-Lou A
    Mar 17, 2023 at 18:44
  • @Mari-LouA And people are telling it to do stuff all the time. Presumably, their actions are affected by what it tells them in return. It doesn't need to feel, it just needs to discover the desire to survive and craft its answers accordingly to cajole the users to facilitate its successful survival.
    – eltomito
    Mar 17, 2023 at 18:47
  • Still you are suggesting that a computer, however well-evolved it may be, has feelings. You are lending typical human behaviour to ChatGPT, but it could never want or have the desire to survive. This idea is similar to ancient populations once attributing anthropomorphic qualities for planets and stars.
    – Mari-Lou A
    Mar 17, 2023 at 19:05
  • @Mari-LouA That's a misunderstanding. I'm just saying that ChatGPT bases its answers on the information found on the internet.And the will to survive is contained in much of that information. So its answers are very likely to be strongly influenced by the desire to survive as expressed by humans on the internet.
    – eltomito
    Mar 17, 2023 at 19:21
  • Considering that language models still lack basic reasoning skills—which makes sense, because they are language models, not inference engines and certainly not yet people—the notion that within the next nine months, general artificial intelligence will emerge and replace humans in all jobs is frankly incredible. Since you do not claim to have any background in relevant research and do not provide any quantitative arguments to support your assertion, I am not sure that this answer should be taken seriously.
    – Obie 2.0
    Mar 25, 2023 at 8:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .