42

It's really quite unbelievable, but artificial intelligence has recently gotten to the point where, given a question, it can give a stunningly coherent answer almost indistinguishable from a real human.

But there are several problems with this:

  • It's not something that you wrote so you can't pass it off as your own work, even under the old rules. See our Help Article on plagiarism. While the AI that generated it could be attributed, the nature of AI is that it breaks down existing writing and reconstructs it, so it's not that simple to name a source.
  • The responses may be stunningly coherent, but they're often wrong or misleading. Dangerous even: We intuitively expect someone fluent in English to be able to answer a lot of questions about English correctly, but AI breaks this pattern.
  • Using the AI requires no particular expertise in any subject. Using AI to write answers that are upvoted means that you're getting credit for something you might have zero knowledge of, which flies in the face of the Stack Exchange model.

As a result, we (the ELU mod team) have decided to ban AI generated content. Don't post it! We have ways to know what you're doing, and there will be consequences.

If you think you've discovered AI generated content, please flag for moderator attention, preferably including the reasons you think it's AI.

About Chat

This ban covers the main site — questions, answers, tag wikis, etc. It's fine to continue sharing funny quotes from ChatGPT in chat. (On the other hand, unfunny quotes...)

Relevant

38
  • 1
    I was not aware of these AI generated answers. Given the loads of LQ answers we regularly get, it is pity to ban AI tout court. Is still possible to use AI as a source for elaborating an answer?
    – Gio
    Commented Dec 10, 2022 at 10:01
  • 14
    @Gio No, for the reasons stated.
    – Andrew Leach Mod
    Commented Dec 10, 2022 at 10:29
  • 1
    The formulation 'questions, answers, tag wikis, etc.' leaves it ambiguous whether the ban applies to comments. Also, wouldn't it be better to specify the 'consequences', instead of saying vaguely 'there will be consequences'?
    – jsw29
    Commented Dec 10, 2022 at 16:32
  • 4
    @jsw29 I wouldn't consider comments to have the same level of importance as questions, answers, tag wikis. The usual policies do apply for comments; they are temporary post-it notes. Do not, however, use AI-written comments and pass it off as own words.
    – NVZ Mod
    Commented Dec 10, 2022 at 16:41
  • 7
    This post is no good. There's no policy here to downvote posts that promulgate information that we know, scientifically, to be incorrect. There are many posts, even by users with a lot of rep, that are upvoted because of style and not because of evidence or reasoning. Aspects of language that are empirically decidable are regularly taken to be matters of opinion and construed as such. The post-truth era has reached EL&U stack exchange. Unless people downvote answers that are rubbish but sound good, and those that are not based on reasoning or evidence and still sound good, EL&Uis doomed.
    – Araucaria - Him
    Commented Dec 12, 2022 at 23:59
  • 16
    @Araucaria-Nothereanymore. The policy "downvote something that's incorrect" doesn't need to be stated here. It's already a policy; in fact it's baked into the tooltip on the downvote button.
    – Andrew Leach Mod
    Commented Dec 13, 2022 at 8:06
  • 7
    "The responses may be stunningly coherent" - syntactically, but not semantically. They're often stuffed with contradictions & non sequiturs
    – Scortchi - Reinstate Monica
    Commented Dec 14, 2022 at 14:11
  • 5
    @Mazura: Suppose you know what a right answer might be: if an 'answer' from ChatGPT happens to touch on the nub of the matter, it's easy to notice only that when you skim it, and not to notice the flaws that become apparent on a close reading. It's treacherous.
    – Scortchi - Reinstate Monica
    Commented Dec 14, 2022 at 16:34
  • 2
    "If you think you've discovered AI generated content, please flag for moderator attention,"... going back like 4 years I have a fairly good record here at meta at detecting hidden spam via URL and IP address, but this seems to go way beyond the skills of the best of the casual user...how are we to detect it ? Either we are users, or we are detectives...
    – Cascabel_StandWithUkraine_
    Commented Dec 19, 2022 at 23:54
  • 3
    @Cascabel_StandWithUkraine_ That part there was a more positive way to tell people to stop arguing publicly over who's a robot (I've seen it happen and it's not nice — a flag is the way to go, even if it's completely wrong). I've been purposefully vague on how we're finding AI content so that any bad actors don't abuse it. The discussion is mainly happening on chat with both mods and trusted users from a variety of sites (sorry, I know you don't like to use chat).
    – Laurel Mod
    Commented Dec 21, 2022 at 19:43
  • 2
    I can't believe that citation nonsense made it into the official help. That is an indication that the people writing the policy have no idea how the GPT stuff works. I know how to calculate the circumference of a circle; Do I need to provide a list of every book I've read that has that information in it, or can I just pick a credible source that supports what I've written regardless of whether that is the book I actually learned it from?
    – ColleenV
    Commented Dec 21, 2022 at 20:33
  • 1
    @ColleenV I doubt you or your (human) source could regurgitate some text you read verbatim unless you're copy/pasting. But GPT-2 has been estimated to do that .1% of the time (as a "very conservative" estimate). GPT-3 may do it as often as 20% of the time. (It took a while to find it, but here's the source.)
    – Laurel Mod
    Commented Dec 23, 2022 at 20:47
  • 2
    @Laurel I do sometimes find myself expressing a concept very similarly to how it was written in something I read. If you went through every book that explained basic English grammar, I guarantee you that there would be a lot of similar sentences. How many ways can you explain what an adjective is? Regardless, if I cut and paste random sentences from across the Internet into my answer, is the problem that I’m not citing my sources or that I’m posting nonsense? The people posting and upvoting garbage is the problem, not the tool.
    – ColleenV
    Commented Dec 24, 2022 at 2:31
  • 2
    @ColleenV If you want to follow that analogy, maybe on the new Titanic it doesn't sink immediately, lots of people die, but it limps along slowly for quite awhile. Rearranging those deck chairs might make things better for the survivors. Anyway, provenance is very important for establishing reliability. But it is a misuse of an LLM to be used for facts (it is not designed to produce facts).
    – Mitch
    Commented Dec 30, 2022 at 16:45
  • 3
    @Mitch I should know better than to use an analogy--it always leads folks to think more about how the analogy is flawed than about the point I was trying to make ;) It is completely irrelevant to everything that there is no way to trace the results of the tool directly back to the source material the model was trained on. The problem is not the tool; all the tool is doing is making it more obvious that SE's system has problems handling content that is low quality but not obviously so at scale.
    – ColleenV
    Commented Dec 30, 2022 at 17:39

4 Answers 4

Reset to default
4

Good luck enforcing it. The problem isn't attribution however. The AI is trained on patterns. As soon as someone creates their own model, the only way you can tell it might have been generated from AI is because it's gibberish.

I could absolutely use an AI to help me write a post and you would be unable to prove I did without exerting more effort than it would merit. It's just a tool. It's like accusing me of plagiarizing a dictionary because my definition of some word is too similar in content to all the other correct definitions. Knowledge can't be copyrighted.

Do we not already punish people for repeatedly posting poor quality content? This fear of AI is silly. The problem isn't the tool, it's how people use it. There is already a policy against posting nonsense as an answer. Whether that nonsense was computer generated so it looks less like nonsense is irrelevant.

The main point of supporting an answer with citations on SE should be to make it credible, not just to give credit. If I copy text from a website generated by a GPT tool and attribute it correctly, is that OK? It is no different from me pasting generated content and identifying the tool as the writer of it. At some point, we need to address the role people upvoting bad content because it seems like it’s correct play in incentivizing low quality content.

12
  • 8
    “This fear of AI is silly. The problem isn't the tool, it's how people use it”. I totally agree with this view.
    – Gio
    Commented Dec 14, 2022 at 10:31
  • 11
    @Gio Nobody is disagreeing with that here, it's a need of the hour to stop the influx of AI answers, until we figure out a better way to deal with it.
    – NVZ Mod
    Commented Dec 14, 2022 at 11:12
  • 2
    AFAIK, this type of gibberish is based on the next word in a string....sounds similar to the newest publications vis a vis Sanskrit grammar decoder businesstoday.in/latest/world/story/…
    – Cascabel_StandWithUkraine_
    Commented Dec 19, 2022 at 20:42
  • 4
    @Cascabel_StandWithUkraine_ You may be thinking of Markov Chains - ChatGPT is much more sophisticated than that sort of primarily computational model. It is trained with reinforcement from humans. There's an interesting point about the limitations of the training from a blog post on OpenAI "...supervised training misleads the model because the ideal answer depends on what the model knows, rather than what the human demonstrator knows.".
    – ColleenV
    Commented Dec 20, 2022 at 16:21
  • "It is trained with reinforcement from humans"...yes I understand that. I think punishment and reward are the terms quite often used by devs to curb AI behaviour. That troubles me...at what point do the AI turn on us like a mistreated junk-yard dog who has not been fed for a while?
    – Cascabel_StandWithUkraine_
    Commented Dec 20, 2022 at 17:02
  • 2
    @Cascabel_StandWithUkraine_ As soon as we anthropomorphize something that is not sentient? When does your hammer get sick of hammering nails and long for something different to do lol? Computers do what we tell them to do; that we aren't very good at predicting the results of what we've told them to do is our problem, not theirs. The breakthrough necessary for a "true" artificial intelligence that can form an intent independent of human interference hasn't been made yet. What we have now are just complex simulations we run under certain constraints.
    – ColleenV
    Commented Dec 20, 2022 at 17:11
  • 1
    "As soon as we anthropomorphize something that is not sentient?"...Interesting. That is in fact part of an article I am working on for substack based on my experiences designing animatronics and robotics in the early 1980s. The premise is that at some point the randomness of the robotic response resembles human response i.e unpredictable. Look...I was hardware and "hard wiring"... I believe you are thinking in higher-level process
    – Cascabel_StandWithUkraine_
    Commented Dec 20, 2022 at 17:22
  • "When does your hammer get sick of hammering nails and long for something different to do lol?" When is the last time you hammered some pieces of wood together? There is a definite chaos in the striking of iron upon iron penetrating pieces of wood. When the hammer slips and we hit our thumb, maybe that is a statistical possibility, but possibly it is an argument between the hammer and hammered? Many cultures support the idea of a spirit in a physical device. BTW..how are your dogs? My girl is doing better.
    – Cascabel_StandWithUkraine_
    Commented Dec 20, 2022 at 18:25
  • 2
    I read an article that postulates that we are all just AI in a very large simulation... ;-)
    – Jim
    Commented Dec 20, 2022 at 20:56
  • 1
    @Jim or brain in a vat, and equally terrifying.
    – NVZ Mod
    Commented Dec 22, 2022 at 4:01
  • Markovian Parallax denigrate... :)
    – Conrado
    Commented Jan 13, 2023 at 16:11
  • 4
    There are good points in this answer. Many people may also be old enough to remember when citing from Wikipedia was treated as plagiarism, cheating, or simply a lack of academic rigor or effort. Many (most?) places now accept high-quality Wikipedia pages as reliable and citable sources, so I see this as part of the natural cycle of any disruptive tech. New norms and knock-on effects will need to be addressed though, but not by fiat.
    – CodeGnome
    Commented Jan 25, 2023 at 17:27
3

Your first dot-point together with comments below the question suggest that AI-generated material is not permitted on EL&U (main) even as attributed sources.

If we find something on the internet that looks like a good illustration of an answer we are putting together, what standards are we expected to apply to ensure we're not citing AI-generated material?

2
  • 8
    Unless you're copying directly from Chat GPT (like all the answers I've seen so far), it can be difficult to know if it's AI generated content even if you have some experience with Chat GPT. I would handle the situation you're talking about differently because it'd be clear it wasn't on purpose (e.g. editing out the offending quote).
    – Laurel Mod
    Commented Dec 13, 2022 at 19:20
  • 1
    Are you citing it or copying it? You can't really cite it in the way you can a publication, website, video, interview etc because AI generated material is ephemeral.
    – hippietrail
    Commented May 5, 2023 at 9:46
2

Stack Exchange changes their mind: “Do not ban AI answers”

31 May 2023

Judging by the most recent policy diktat announced by Stack Exchange, AI generated answers are no longer to be unilaterally deleted, and users who make use of this tool will no longer be automatically suspended. There are caveats but generally speaking, answers sourced by AI Chat GPT are not forbidden by the company and moderators are encouraged (is that the right word?) to refrain suspending users who are only suspected of copying several answers from LLMs (Large Language Models).

What is the network policy regarding AI Generated content?

A few of the most significant changes in policy are listed below. The emphasis in bold are mine.

Finally, internal evidence strongly suggests that the overapplication of suspensions for AI-generated content may be turning away a large number of legitimate contributors to the site.

In order to help mitigate the issue, we've asked moderators to apply a very strict standard of evidence to determining whether a post is AI-authored when deciding to suspend a user. This standard of evidence excludes the use of moderators' best guesses based on users' writing styles and behavioral indicators, because we could not validate that these indicators are actually successfully identifying AI-generated posts when they are written.

and further…

We've reminded moderators that suspensions (and typically mod messages as well) are for real, verifiable malfeasance only, and should not be enacted on the basis of hunches, guesses, intuition, or unverified heuristics. Therefore, we are not confident that either GPT detectors or best-guess heuristics can be used to definitively identify suspicious content for the purposes of suspension.

How moderators, or anyone else for that matter, is supposed to verify whether an answer has been entirely or partially copied from ChatGPT has not been communicated.

For example, before flagging content which was likely copied and pasted from ChatGPT, I would first check the author's previous posts. If their previous efforts were, at best, written in broken English or worse, semi-literate, it was a strong indicator that this new content–always in flawless English and always unattributed–was not original. It goes without saying, this method was impractical for new contributors to the site; however, if their content consisted of multiple answers posted in the space of an hour or little more, it was a persuasive gauge that the answers could not be original.

To clarify–as far as I'm aware–it is impossible to Google search content generated by ChatGPT because this content does not appear in searches. Moreover, AI detection tools simply do not work. A few hours ago I copied and pasted two answers by two different users, answers which were deleted by EL&U mods because copied verbatim from ChatGPT, and the AI detector tool–which is itself powered by AI–confidently said that these two answers were 98% and 99%, respectively, created by a human. But both users were suspended for a month, and since then one has deleted their SE account while the other has not returned.

P.S. You have to create an account at the AI detector website if you want more accurate results. I chose not to.

11
  • Can you explain your last para? You posted two answers and others were suspended? I just don't understand that.
    – Lambie
    Commented Jun 1, 2023 at 15:54
  • 1
    @Lambie - pasted not posted - I assume she copied two deleted answers into a chatgpt detector which said they were human-generated even though they were deleted as AI content. Or something like that.
    – Heartspring
    Commented Jun 1, 2023 at 16:15
  • @Heartspring Why were the users suspended if she was the poster?? I still do not get it.
    – Lambie
    Commented Jun 1, 2023 at 17:09
  • 1
    @Lambie - two random, now suspended users, posted answers suspected to have been made by chatgpt. Mari-Lou, with her 'view deleted posts' privilege copied those answers into an AI detector. The detector seems to have said that they were human made.
    – Heartspring
    Commented Jun 1, 2023 at 17:27
  • @Lambie I copied and pasted two answers by two different users that had been deleted by the EL&U mod team.
    – Mari-Lou A
    Commented Jun 1, 2023 at 17:48
  • Well done, and thank you @Heartspring, it's exactly as you described. Hopefully by adding "copied" in the edit this clears up any confusion
    – Mari-Lou A
    Commented Jun 1, 2023 at 17:51
  • How perceptive I am :) And yes, I think your edit is a good clarifier.
    – Heartspring
    Commented Jun 1, 2023 at 17:58
  • Predictable outcome of the new reform, see: meta.stackexchange.com/questions/389742/…
    – Mari-Lou A
    Commented Jun 2, 2023 at 11:11
  • 1
    Half the SE moderators are now on strike. On SE Biology others of us who are not mods are supporting them by non-participation. What is the position of the mods on this list?
    – David
    Commented Jun 12, 2023 at 21:53
  • @DavidstrikingagainstAI so far there appears to be either deference or silent complacency, which is weird considering they fully supported banning the use of GPT, and were lightning quick to suspend users, suspected of dishonesty, for one month. To be clear, my position was of compromise, if unoriginal content is always attributed, and AI answers are supported by research and the authors' own expertise, the two should be able to work in harmony. Unattributed sources is plagiarism, failing to support answers with facts and references should raise red flags and be dealt with accordingly.
    – Mari-Lou A
    Commented Jun 13, 2023 at 7:19
  • 2
    We all have personal decisions to make on this question. I can understand if the mods here do not wish to make individual statements because they do not wish to sour relationships with other mods. However one’s own attitude no longer becomes an academic question when the mods are striking. Normally I regard with disdain changing one’s handle to express some political viewpoint as it has nothing to do with the list and is only so much “virtue signaling”. But in the present situation posters should be made aware that both moderators and active contributors are striking. And why.
    – David
    Commented Jun 13, 2023 at 21:41
1

Is it ethical for an answer that is clearly attributed to AI source, but which also includes original content be deleted? Can we not trust community members to know how to cast their votes?

One such answer was deleted, and although the author's conclusion wasn't particularly elegant, it made a more than valid observation.

  1. Step forward ChatGPT!
    I typed the following into ChatGPT:

Is there an antonym for the term "et seq"?

What followed was ChatGPT 3's explanation of the meaning of “et seq” and how it is used. It also provided "et ante" as a possible antonym but added it was not commonly used in legal settings and

it may be clearer to simply specify the range of pages or sections being referred to explicitly, rather than relying on a Latin abbreviation.

As well as “et ante”, the author also suggested Et qui prius and Et unus ante and concluded

I do not know of any established usage of either of those two Latin phrases. Therefore, to put it in colloquial terms, you would look like a right prat if you tried to use a Latin antonym of et seq.

If the AI source is attributed right at the beginning, and the author does not rely on exclusively on ChatGPT but cites a second source (e.g. Google Translator) and adds their “original” conclusion; on what grounds should that answer be deleted?

How many answers posted on EL&U have consisted of only dictionary references, lengthy excerpts copied verbatim from Wikipedia, citations from disreputable sources e.g Urban Dictionary, and lifted answers from blogs by individuals whom we know nothing about? As long as these sources were attributed and the answer had the occasional link, we–the community, were allowed to judge for ourselves the quality of the answer.

10
  • 5
    Using AI-generated content, even when duly attributed, would clutter the site with answers of dubious quality and so dilute the usefulness of the answers that come from those who actually know the subject matter. People who have genuine knowledge that is worth sharing with others on this site don't need to consult the AI. It is true, though, that the answers of the let-me-look-it-up-in-a-dictionary-for-you kind also fail to manifest genuine knowledge of their subject matter.
    – jsw29
    Commented Apr 2, 2023 at 15:52
  • 1
    @jsw29 but what about the users and their votes? Is every user on EL&U a subject expert? Many are but not all. Discovering whether et ante is Latin doesn't necessitate a knowledge of Latin, and what about the author's original, albeit slightly crude, conclusion? In fact, the answer attracted a few upvotes which was then subject to a number of downvotes. Copied answers from linked sources are permitted, answers that contain no original content are permitted, answers that offer no supporting evidence are permitted yet an answer with two named sources and an "original" solution must be deleted.
    – Mari-Lou A
    Commented Apr 2, 2023 at 16:25
  • 2
    Passing off an AI generated answer as your own is plagiarism, and lazy. But if the AI generated text is part of a longer, more detailed, and correctly attributed answer, why should we ban these types of answer? I understand if it is producing code; there must be some form of verification that the code works, and that is time-consuming, and unfair to place this burden of responsibility to the entire community but this is English. We don't produce code or fix coding errors. That's a different ball game.
    – Mari-Lou A
    Commented Apr 2, 2023 at 16:29
  • @jsw29 By the way, if it's of interest, ChatGPT has recently been banned in Italy due to privacy and safety issues.
    – Mari-Lou A
    Commented Apr 2, 2023 at 16:34
  • 7
    The gist of my argument is this. AI-generated content can never be sufficient on its own; it needs to be evaluated by a knowledgeable human (I think you agree with this step). But, a human who has enough knowledge to evaluate AI-generated content, can just write an answer on the basis of that knowledge, without consulting the AI; the AI cannot add anything to an answer that a knowledgeable human can produce. There is thus no downside to banning all AI-generated content. The fact there is a lot of human-generated junk on this site is not a reason for allowing AI-generated junk as well.
    – jsw29
    Commented Apr 2, 2023 at 20:30
  • 1
    It's quite fine to remove answers that are tainted by machine-generated content. If people want LLM, let them ask it themselves. But here, let humans answer the questions. Although you're right about the verbatim copying problem: my highest-voted answer is to a SWR with just two copied chunks, no expertise and very little effort required. Still, let it be a person compiling the chunks, with non-virtual intelligence.
    – Conrado
    Commented Apr 3, 2023 at 2:15
  • 2
    @Conrado A person googling has to choose from many hundreds maybe even thousands of hits whereas one who asks an LLM has but one answer. What is there to stop the person checking that the information is correct? And what stops that person from changing the wording, expressions, even rewriting the entire thing to match their personal style? This banning ChatGPT will not get easier over time.
    – Mari-Lou A
    Commented Apr 4, 2023 at 7:51
  • 2
    Nothing to stop them, I guess, but checking that the information is correct and rewriting the entire thing sounds like as much work as writing my own answer the old way, and I'd still owe credit to some faceless LLM. I mean, let people ask C-GPT if they want, but make them work its information into their own answers; and punish stochastic parrots severely. No, it won't get easier, but you asked about ethical! Anyhow, that's my strong feelings about it. Cheers!
    – Conrado
    Commented Apr 4, 2023 at 14:06
  • 3
    While I agree with the conclusion of that answer ("don't use Latin"), it's not a good answer; Google Translate is really just as bad of a source. So far, I have only seen one other answer on ELU that had attributed AI content, though that was edited out by the author in response to downvotes. I don't think we're losing much of value. The possibility of bad answers from other sources shouldn't play into this — it's a different problem. Also, a blog or UD is one person's opinion (which can count for something), while ChatGPT is no person's opinion.
    – Laurel Mod
    Commented Apr 5, 2023 at 2:43
  • @Laurel I understand why the deleted post is not a good answer, but it is an answer and it contains the author's opinion. If you were to delete the segment that is copied from ChatGPT, you would still have an answer. What is the purpose of awarding the privilege of downvotes to users if every bad/poor or factually incorrect answer should be deleted?
    – Mari-Lou A
    Commented Apr 5, 2023 at 10:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .