Does ELU Have Worse Questions Than Other Sites?

Question

This is a follow-up to a couple comments on What can we do to make this site more “intimidating”?

@Jason-C asserted that ELU's problems with question quality are no different than what other sites face. Specifically:

Many sites on SE have faced similar problems and solved them, never by making the site more "intimidating", but instead by using the tools we have been given appropriately.

Is that really true? I'm no SE guru, but I do pop over to other sites and have not noticed this same phenomenon there. Not even close. Writers, Sci-Fi, RPGs, WorldBuilding... sure, they get silly questions now and again, but the vast majority seem to be good ones.

I likewise find it hard to believe that an academic-oriented site like "MathOverflow" ever got the same deluge of horrible questions as we do. It would be like someone going there and asking "What is 27 x 52?" To which the immediate response would likely be: "Is your calculator broken?" or "Why in the world are you asking that here?"

I like @Andrew-Leach's summary of why ELU may be different:

I do think that subject area to be a special case, because the world his wife and his dog think they know something about English (as opposed to car maintenance, or mathematics), but members of other communities may not understand that.

But I'm open to being convinced that I'm wrong. I'm not savvy enough with the data stats to answer my own question, but perhaps someone else is?

---

Edit to clarify why I'm asking this:

Imagine you're complaining about your house being swarmed by flies and your neighbor says, "I had that same problem. Just use a flyswatter." But you've been using your flyswatter, and it's not working, and your arm's getting tired.

Before you start asking for a bigger flyswatter, I think you need some justification for why you need one when nobody else does. Maybe you don't. Maybe you're just doing it wrong. Maybe you're overstating the fly problem. Maybe your neighbors know something about fly-killing tactics that you don't.

Then again... maybe you've got a whole fly colony and nobody else does.

This is a question I believe can be answered with real data.

Answer 1

The programmers site has a similar problem. Someone suggested that it might be due to the name of the site. The specifics are different, but the idea remains that the community's name may be attracting certain kinds of content. There's a lot of content that fits under the heading English Language and Usage. (NB: I'm not proposing a name change for ELU - such a drastic step has repercussions, some of which were brought up in the discussion on the programmers site, including rebuilding site reputation / brand recognition and SEO ranking.)

Yes, there are rules about what's on-topic and off-topic, but to a new user landing on an ELU page, the link to those rules is on a page that itself is accessed via a link consisting of one word in the footer, a place many websites put their copyright notices, names of web developers and other things not of primary interest to most site visitors. That is, visitors don't normally intentionally visit the page just to read footers, much less documents linked in the footers.

Even the HOW TO ASK box on the Ask a Question page doesn't tell you immediately what's on-topic and what isn't. The link to the help page is more visible, but the relevant information is still hidden behind a link to 'help'. People who land on the site and want to ask questions probably consider the 3-box form (title, body, tags) simple enough that they don't need help. A dedicated link named "What questions does this site accept?" might be more useful if you wanted visitors to look it up.

Having said all that, as a regular visitor to ELU, I haven't (yet) found the low-quality problem to be particularly problematic. It's easy enough to just skip uninteresting questions. If more is desired, consider requesting a tag exclusion filter.

tl;dr: On the "Ask a Question" page, link directly and prominently to the on-topic / off-topic rules.

---

Update related to @michael_timofeev's answer.

Michael Timofeev pointed out the gentler, more helpful approach taken by users of the mathematics forum. Would it really be so bad if ELU followed the same path? We'll get more me-too questions, but these would be pointed to canonical questions and answers. With the extra attention on the canonical pages, they may eventually migrate to wiki status. If the concern is losing the canonical Q&A in the sea of lookalikes, consider also that linking back from the (possibly many) me-too questions to the canonical Q&A should help improve the page rank of the canonical Q&A / wiki. Far from diluting the usefulness of the forum as a place to find answers, allowing duplicates of popular questions automatically make it easier (via improved page rank) to find the canonical Q&A.

Answer 2

I appreciate everyone's input as to why ELU may be worse than other sites. I was actually looking for proof/disproof as to whether it actually was or not. I believe I have figured out the data explorer well enough to answer my own question.

This graph shows the percentage of closed questions (compared to asked questions) from 9/1/15 - 1/3/16.

As you can see, Programmers and ELU rate significantly higher than other sites - both needing to close about 28% of questions asked. (On some months, ELU was as high as 48%.)

Edit to add: As curiousdanni notes in the comment, this doesn't include deleted questions, so the actual number of bad questions is even worse.

This does not seem to be tied directly to popularity; Mathematics had 10x as many questions and StackOverflow had over 100x as many questions, yet both have much lower close rates.

It's not a perfect representation of question quality, but I think it illustrates that the problem is not unique but it is likely much worse than average.

Answer 3

I don't have hard comparative data, but I suspect we do have a problem most other sites don't (or at least not to this extent): a lack of functional consensus on the purpose of this site.

If you've seen me on Meta much then you'll know I often complain about high-rep users who never vote, edit, flag etc, and just answer questions. Other people always respond and say that our users have the freedom to decide how to participate, which is fair, they do. But the reality is that this site has two groups of people with very different expectations of what the site scope is. The group that frequents Meta have come to the consensus position that questions without evidence of research, proof reading questions etc are off-topic. But there is another group which deem any question coherent enough to understand worth answering.

Which other sites are so divided over whether homework questions are allowed?

Now the bad questions inevitably get closed, even if they do get answers from such users first. I'd like to say that having so many users with a different position on scope sets a bad precedent, encourages those questions to be asked etc, but I have no evidence that stopping those answers would really help things much at all. But I think it does affect our moral, for me at least. Our problem isn't just that we're fighting a never ending flood of bad questions, but that there are dozens of 5000+ rep users who aren't joining the fight, and some are even actively working against us.

Answer 4

My theory: ELU has it worse because it has been successful.

Unsuccessful sites (like Travel.SE) mainly get users from other Stack Exchange websites (primarily Stack Overflow), who are familiar with the Stack Exchange model, and know how to ask good questions.

Successful sites get people coming from Google. They may or may not know how to ask good questions.

ELU has been a successful site. To make it worse, ELU is a non-STEM site, so probably has less overlap in the user base with other Stack Exchange sites.

Answer 5

The creation of ELL was supposed to help alleviate this problem, but it seems like ELU gets as many novice/basic/learner questions now as it ever did. Many of these come from new users who appear to be English learners (or are English learners – one recent question began with First of all, I am a french secondary school pupil so I am sorry for every mistakes). However, such questions are rarely migrated, and I don't see many comments pointing toward ELL, either, unless I'm the one leaving them.

Questions like this one may ultimately get (rightly) closed, but, from what I've observed, they get routinely answered and upvoted as well. It seems like a critical mass of English learners have made ELU their Stack Exchange of choice for honing their English skills, and more are finding this cite every week.

I often see brand new users ask a question like:

What's the difference between specify and point? They have the same translation in my language.
_{(emphasis added)}

and they are exhorted to go find a dictionary, rather than the English Learner's exchange. When that happens, I wonder if maybe Robert Cartaino was right, and we should have just kept all the English questions in one place.

As it is now, it seems like we have one community specifically geared toward learners – and another with a hodgepodge of learners and serious etymologists jockeying to hold sway over its future.

Answer 6

I think Andrew Leach hit the nail on the head.

If we look in other areas that I visit things seem to be a bit different. For example, on the mathematics forum most of the questions and answers are handled in a different way. Take this question, https://math.stackexchange.com/q/679712/262555.

It is a question which could be googled or looked up in a third semester calculus textbook. However, no one told the person to "look it up." Users pitched in to help. Of course this is not a 27X52 type of question. It is the kind of question that requires knowledge and the ability to teach mathematics (or at least give an answer that can help someone learn.) In most math, it's not really possible to say "in my opinion," because either the answer is correct or not. The way to get to the answer varies, however there are usually tried and true methods that people prefer because they save a lot of time and confusion.

On the math forum there are questions that amount to "proofreading." In fact a large number are of the sort "I'm trying to prove this and so far I did this. Am I on the right track?" Many of those questions are also ones that could be "looked up on the internet." Of course people don't because they enjoy the community feel of "doing math." and math requires a bit more help and "seeing the big picture."

I have posted questions and had them ignored or had my butt chewed out because they were "basic" or you could "look it up on the internet." In the math department "basic" questions such as "why is a negative times a negative positive?" have been discussed and rediscussed many times, and I have seen a number of brave dads ask some basic questions to help their 8 year old son. The brave dad got answers, and they weren't "look it up on the internet." or "Did you ask his teacher for help?" I saw a bunch of people vying to give the coolest answer and add something new to an old question.

Contrast that with ELU where anyone who speaks English can ask a question or answer it, and you can right away see the potential for "trouble." Anyone can answer a comma question based on "in my opinion," "according to Dr Bund..," "Who knows because there are no rules..." or "Grammar Girl states that..." Heck, even I can answer questions around here. Combine this with how "stirred up" some of us seem to get by "wrong" answers and the fact that some people around here have "seen it all" and there are going to be questions about the site and our common purpose. How can we have a common purpose when no one here can really agree on where commas should go in a sentence? How can we have common purpose when some users pop in every once in a while to answer a few questions for the heck of it, some stop by to "clean up the trash" and then leave not to be seen for a few weeks, and other users want this place turned into the hallowed halls of academia where everything has to be footnoted and people walk around with robes and mortarboard with a site-approved tassle.

It might be time to realize that because language is such an open field and it does not take a college degree to answer questions, this site might always be "plagued" with this "problem" and that it will have to undergo a periodic "purge" to rid it of a nuisance inherent in it's structure from the get go. On the math site it's harder to fake it and mathematics from the get go is intimidating so you aren't going to get very many people asking 2+2 sorts of questions. Contrast this with English where someone who listens to Grammar Girl feels they are a grammar expert, someone who is a computer programmer feels they have a "structured mind" and can answer questions, a guy who reads "tough books" in his off time can weigh in on relative clause matters, and someone studying English in Oz needs some quick help on his homework, and you have a recipe for disagreement...or maybe you have a recipe for a site with a lot of variety and no possible way of having consent.