2

Having successfully transitioned from being an English language learner to an appreciator of the language, I am part of both the ELL and this stack exchange. And I am proud to be part of both vibrant communities.

However, in the recent days, I have noticed an increasingly large amount of question about basic syntax and vocabulary, that are clearly better addressed in the ELL forums. While the mods have been great at their migration duties, it is not enough; they are mere mortals after all, and there is a massive influx of these questions.

I propose an automated system to detect whether this question could be an ELL worthy question, or an English Stack question. How would this be done? Well, there are key terms that tend to appear more frequently in one area than the other. This might not have to do with the complexity of involved words, or the correctness of the user's syntax (although those could be secondary decision features). Take the word parachute. An English learner might ask about its parts of speech, and its meaning/ definition. An appropriate question here would be concerning the historical phonetics of the word, or its etymology and evolution. These key words are more likely to be in questions relevant to this site. We can train our AI model on sample questions.

No system is perfect, but we need a way to sort the questions before they are out, and not after. Then we can turn our focus to truly enjoying the nuances that the English language offers for us here.

Cheers

25
  • 3
    What does ELL think of this proposal?
    – ab2
    May 10, 2019 at 0:01
  • 6
    Lets not send those questions to ELL. If you can make such an AI system, use it to tell people to go read a dictionary or take an English class.
    – curiousdannii
    May 10, 2019 at 1:09
  • Humans do it better.
    – Mari-Lou A
    May 10, 2019 at 6:10
  • 1
    This question is very simple, how would any automatic system identify it as such? There's a risk that the software ignores typos and grammar error and fixates on typical phrases and expressions used by non native speakers, then we're entering discriminatory territory. I hope this AI system, which some users might think is the solution to LQAs, doesn't see the light of day.
    – Mari-Lou A
    May 10, 2019 at 6:18
  • 1
    The AI doesn't have to migrate stuff immediately; it could just strongly suggest doing so or auto-flagging it for human mode review.
    – Confused Soul
    May 10, 2019 at 9:03
  • @Mari-LouA For example, the "should I say this or that" is unlikely to appear in too many ELU posts. Nothing is flawless, but we need a system to preserve the integrity, and identity of both sites.
    – Confused Soul
    May 10, 2019 at 9:04
  • @ab2 If it does not generate sufficient debate and interest here, there is no point taking it to ELL.
    – Confused Soul
    May 10, 2019 at 9:06
  • 1
    Two questions of mine using the words "should I say...?" english.stackexchange.com/… worthy of ELL?
    – Mari-Lou A
    May 10, 2019 at 10:06
  • I agree with you. Such appearances might pose difficulty to a software. However, also note the use of the words "history" and "origin", which are more likely present in questions here. And you can find tons of ELL questions with the words history and origin, I'm aware, but
    – Confused Soul
    May 10, 2019 at 11:46
  • 1) The model will be based on a computer analysis of questions and answers, not on any standards that I or anyone else sets
    – Confused Soul
    May 10, 2019 at 11:46
  • 2) I don't intend it as a definite means of classification, just a tool that narrows the search down for our hardworking mods
    – Confused Soul
    May 10, 2019 at 11:47
  • 3
    @DanBron Detecting “ELL questions” is not the same thing as detecting questions that should be migrated. We have had multiple instances of people insisting that a simple looking question be migrated from ELL to ELU because it turned out to be more interesting than certain folks expect of questions from learners. I would hate to see something that looked simple in pattern but raised interesting questions for serious English enthusiasts automatically put into a rejection queue subject to band-wagoning.
    – ColleenV
    May 10, 2019 at 16:03
  • 1
    @JJJ do machines care? Do machines make sacrifices for their families? Do machines make decisions based on what is "right" or "wrong"? Do machines think? Do machines spontaneously create art? Do machines laugh?
    – Mari-Lou A
    May 12, 2019 at 5:19
  • 2
    We are much more than complex neural networks..
    – Mari-Lou A
    May 12, 2019 at 5:23
  • 2
    @Mari-LouA your intuition is correct. The ideas that artificial neural networks are just simplified or under-trained representations of the human brain are just wrong. The rules by which our neural networks are constructed and our neurons trigger are encoded in our DNA (not the nets themselves) so our brain’s connections and “firing” won’t be the same even if our environment is. Humans forget and misremember things, but not randomly or disjointedly. Then there are things like Glial cells. We can use some observations we make to solve problems, but we don’t really understand human cognition.
    – ColleenV
    May 13, 2019 at 17:29

2 Answers 2

Reset to default
3

Questions should only be migrated if they are both off-topic for the site and on-topic on another site, so not only would the AI have to detect whether it was off-topic here, it would need to detect whether it was on-topic on ELL.

ELL tolerates questions with less research because it's hard to ask a question in a language you're still learning. It's a lot easier for us to help learners get their question into shape if they ask directly on ELL, because putting a migrated question on hold rejects it.

A better use of AI would be detecting whether someone asking is likely to be someone learning English as a foreign language, and then leaving a comment directing them to ELL where their questions might be better received.

What's the point of migrating a low quality question? Migrations are really confusing for new users, especially if they don't already have an account on ELL. Migration should be used for questions where there is something of value to preserve, like clarifications in the comments, or where the question has plenty of context and detail, not for one line "which word do I use?" questions.

Migrations, in my opinion, should never be done without human involvement. The rules aren't clear enough that we can get people to agree more often than not on what should or shouldn't be migrated or closed, and standards change as the community changes. Computers are not good at judgement calls and interpretation; they're good at detecting patterns and there's no clear pattern for questions that are off-topic on EL&U and on-topic on ELL.

I created a query to look at ELU to ELL questions. If you set the minimum to -50 you should get all the migrations that haven’t been roomba’ed yet. I like to set the minimum to 3, which represents a decent amount of engagement with the question by the ELL community. The default start date is when the migration path opened, but more representative results would probably be restricted to just the past year. Be sure to run the query against ELL. The default is Stack Overflow, and we don’t migrate many questions there.

https://data.stackexchange.com/ell/query/485239/migrated-questions-from-elu-with-scores-less-than-parameter?StartDate=09%2F22%2F2015&MaximumScore=50&MinimumScore=-50

In the past year, we had about 613 migrations and only 17% of the migrations have a score of 2 or higher after migration. I don’t think that is a great success rate, but not terrible considering that only 2.8% were scored negatively. The stats include rejected migrations and those closed as duplicates (that’s not a rejection). I’d have to do more work to sort out the rejections.

12
  • Presumably the ML method would be trained on questions that were 1) successfully closed to be migrated to ELL -and- 2) accepted on ELL (or not subsequently closed there as offtopic). The whole point would be to use criteria that would make both sides happy.
    – Mitch
    May 10, 2019 at 13:48
  • Not that bad m in favor of the process. If such items were put in a special queue for humans to have final say then maybe. But then why stop at 'migrate to ELL'. Train it for all close reasons.
    – Mitch
    May 10, 2019 at 13:50
  • @Mitch I don’t agree that if a migration wasn’t rejected, it was successful. There are many migrated questions on ELL that are nothing but noise (but overall migration is a positive right now). Automating migrations/reducing their cost is unlikely to improve anything. Automating low quality closures would be better. We can’t close every question here that merits it, so it’s unclear to me why anyone would think that ELL could manage to reject every poor migration.
    – ColleenV
    May 10, 2019 at 14:03
  • I thought a migration to ELL had to be approved by ELL people somehow. If the migration is automatic, then yes that is a bad training method. Possibly train then on what is in general closed in ELL. The process should always take into account what is good for ELL.
    – Mitch
    May 10, 2019 at 14:15
  • I get some of the points argued against automation. However if you check the recent questions on this site, you will, at any given time, find a large quantity that are unsuitable for ELU. Sometimes, they are poorly formulated, even for ELL questions, but the fact remains that no modification can make them appropriate for this site. The final say can be human, but I see it is urgent to introduce a system that systematically identifies such quesitons.
    – Confused Soul
    May 10, 2019 at 14:22
  • 1
    @ConfusedSoul unsuitable for ELU does not mean they’re suitable for ELL. If they’re off-topic, close them.
    – ColleenV
    May 10, 2019 at 14:24
  • @Mitch most rejected migrations that I’ve seen should have been closed here for lack of research/context/detail. I have a SEDE query that can be used to dig into migrations and how they’re received. Let me see if I can find it.
    – ColleenV
    May 10, 2019 at 14:26
  • @ColleenV {"captcha":true} is what I got form the SEDE. Any idea on what I should do? Also, I am aware that unsuitable for ELU doesn't imply suitable for ELL, I'm focusing on the particular class of questions and influx of many new users who seek the ELL site, and must be guided there as effectively as possible.
    – Confused Soul
    May 10, 2019 at 15:22
  • @ConfusedSoul log into data.stackexchange.com first then go to the url. If that doesn’t help, it’s probably browser settings? I disagree that migrations are the most effective way to guide people to ELL, particularly automated ones. Migration is very confusing and non-intuitive for new users. The most effective way is a kind comment explaining why they will get a more helpful answer on ELL. A comment allows them to ask questions of the person guiding them so they better understand why ELL is a better match.
    – ColleenV
    May 10, 2019 at 15:38
  • Migrations seem to be overwhelmingly well received. As the situation stands, I can't see any helpful comments guiding these users. In the past 6 hours for example, we have three questions that can be reformatted for ELL (I admit they are poor as they are, but they can never work here, any improvement will send them to ELL). Links are: english.stackexchange.com/questions/498014/… english.stackexchange.com/questions/497988/…
    – Confused Soul
    May 10, 2019 at 21:17
  • Finally, our AI can be instructed to provide helpful comments and guidance to send them to the ELL; it does not have to be a faceless bot of exiling doom :p
    – Confused Soul
    May 10, 2019 at 21:17
  • 4
    ELL has always said "do not migrate crap" or words to that effect, with AI that would become inevitable. Even if candidate questions were reviewed for migration, we all know that 97% of users in the review queue would agree with migration.
    – Mari-Lou A
    May 11, 2019 at 6:48
-1

After reading the link "https://data.stackexchange.com/ell/query/485239/migrated-questions-from-elu-with-scores-less-than-parameter?StartDate=09%2F22%2F2015&MaximumScore=50&MinimumScore=-50" that Andrew provided, I noticed that such an idea perhaps might be feasible.

I understand many of your concerns about mass-migration and destroying valuable content, however I believe we can work together to set a system that balances precision with efficiency.

To that end, I will begin exploring options to develop a prototype of an ML based classifier (starting with something like Bayesian statistics). Ultimately, the AI will show us what it can offer, and we the community will be the final judges.

Whoever has any suggestions or input, please do comment them. Upvote this answer if you support the development of a text classifying bot.

3
  • 3
    Upvoted. Suggest you turn it into a chatbot like SmokeDetector. That is, it doesn’t take any direct action, simply posts in chat when it sees a close-worthy Q, and let’s the community make a decision.
    – Dan Bron
    May 11, 2019 at 14:02
  • Thanks... I'll see the options for adopting an open source neural network, but I certainly do not have the means or the experience to create one on my own. I'll check the link.
    – Confused Soul
    May 12, 2019 at 9:52
  • The link that Andrew provided?
    – ColleenV
    May 13, 2019 at 1:37

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .