A look at question quality

Question

Table of Contents

This is a really, really, long post. And yes, I hate to create another post on our current "quality" situation, but this post is seriously long to the point where I don't want to distract from anyone else's post. In addition, this represents my personal view only.

In fact, it's so long that I'm including a table of contents to help you read only the sections you want to.

• Previous Questions
• Audience, and why looking at is isn't entirely productive
• Do we have a quality problem?
• What is Jeff concerned about?
• Close reasons
• What is off-topic?
• Question Pre-requisites
• How can we improve? <-- This is the TL;DR one to read

---

Previous Questions

First things first—I'm going to quickly link to the questions here on Meta.English about how we can improve on the recent situation:

One is on whether we have a problem:

• Is EL&U succeeding?

Three are on what type of audience is appropriate:

• Who is EL&U's audience?
• Problem with faq: EL&U audience - experts only or students too?
• Is English Language & Usage highbrow?

And another is an attempt to clarify the on-topic/off-topic policy:

• Can we have some guidance as to what is on- and off- topic on this site?

---

Audience, and why looking at it isn't entirely productive

I think the best way to solve this problem successfully is to take a close look at question quality and close reasons. However, before I do that, I want to address why I don't think questions about our audience are that helpful.

If we take a look at SE in general, none try to scope for audience. Instead, the scope is for on-topic and off-topic, and both beginner questions and expert questions are welcome. In fact, the majority of the answers to those two earlier questions make it clear that this is how most of our users feel—if you have the ability to consult a dictionary and/or thesaurus before posting, your question is probably welcome. In other words, anyone who genuinely wants to learn is welcome—a philosophy that pretty much applies to any SE site. So while it's good that the community is reminded of this, there's a limit to how much use it can be to us.

With that, I'm going to try not to mention audience again. (Ok, actually I do mention it one more time.)

---

Do we have a quality problem?

Let's see if there is a problem. I know this has been discussed, but I'm going to take a non-holistic, methodical approach.

I looked at the last 15 questions asked as of writing, and here's a quick summary as they were originally posted. The details for the questions are at the bottom of this post.

• 6 excellent questions (of which 3 I could not fault, and the other 3 were nitpicking problems)
• 4 ok questions
• 2 not-so-great question
• 3 closed, none as exact duplicate

In other words, 10 were fine, and 5 weren't. Of the bad 5, 1 was edited and is now equal to a ok question, 1 was a list question, 1 was based on an erroneous assumption, 1 was provided without enough detail, and the last 1 was off-topic.

How is that? That number means that 2/3 of all questions posted, examined before editing, are fine—and that the remaining 1/3 isn't. I've had a quick chat with mods from a few other released sites, and they're also running at about that amount. In fact, that's a solid number, and shows that there isn't a rampant problem.

---

What is Jeff concerned about?

Let's take a quick look at Jeff's concerns (And you're free to edit any more you have into this post, Jeff):

• Questions about jokes
  • Jeff thinks that they deteriorate site quality. The community disagrees, I disagree, but Jeff isn't budging. I realize this has caused this entire wave of debate—it's definitely caused several of our members to leave or consider leaving.
  • Ideally, I'd love to see Jeff come around to our way of thinking. However, assuming that that doesn't happen, the compromise is that we allow Jeff this one concession in return for having built Stack Exchange. I feel and hope that the community will allow this, in return for not allowing this sort of drastic action to happen again.
  • How can we stop this from happening again? I suggest a set of clear, objective criteria that we can use to determine on-topic and off-topic—a series of questions/flowchart of sorts. I've expanded this below. If we can get Jeff to agree to also use this flowchart so that it's entirely objective, I doubt it'll ever happen again.
• Questions about things of a perhaps less-than tasteful manner
  • More specifically, the titles need to be clean. As I understand it, EL&U seems to be sanitizing these titles as quickly as it sees them. Jeff wants them even cleaner, but we don't want the titles to be overly vague. I can only suggest we keep the importance of censoring titles in mind when dealing with these sort of questions.

---

Close reasons

I'd just quickly like to go over these:

• exact duplicate
• off topic—should be determined by a set of clear, objective criteria, a proposal of which is set out below
• not constructive—this is the old S&A close reason—one question, one answer (certain list questions of exceptional quality excepted)—otherwise it gets closed

• not a real question

  • if a complete answer to the question is as long as this post because of how broad the question is, something is seriously wrong—in other words, don't be vague and dance around what you're trying to get at
  • in addition, if the question is so basic that it demonstrates a lack of understanding of basic English, such as SVO order, then it's NARQ—does not apply if asking how or why a particular basic form of English came about (this, by definition, only applies to ESL questions)
• too localized—it's also not a real question, but this time because it's far too specific (which, by the way, is not the same thing as detail, which is good)
• general reference—if I consult Wiktionary, Dictionary.com, Thesaurus.com, Google, or Wikipedia (or other appropriate well-known sources), and find the answer just sitting there without any interpretation required and the answer is utterly boring, it's general reference

---

What is off-topic?

In light of the recent debate about questions about jokes, here's a proposal for revising on-topic and off-topic:

This only determines whether a question is on-topic or off-topic. They can still be closed for any other close reason.

• Is your question about English words (e.g. what does a word mean or "word1 vs word2") or phrases (including idioms and the like)? On-topic (covers the existing usage, word choice, etymology, pronunciation, and spelling on-topic reasons)

• Is your question about grammar or syntax, and if so, does it make specific references to what is of concern? On-topic (covers the existing grammar and "problems encountered..." on-topic reasons, while excluding proofreading)

• Is your question about how English varies? On-topic (covers the existing etymology and dialect differences on-topic reason)

• Is your question about punctuation as it relates to English? On-topic (covers, well, duh)

Otherwise, it's off-topic. This does its best to cover the existing /faq on-topic and off-topic things, but improvements are no doubt possible.

---

Question Pre-requisites

I'd like to address one final thing, and that's what needs to go into a question before you should post it. Of the 12 open questions, 6 had research, and 6 lacked it. I don't think that that's a particularly bad number—if you look at the questions themselves, some can't be easily researched, which brings it down to 2 that could actually use research.

However, the question remains of how we address it. I don't see much to do—a question without research where research could be easily done is very likely to have other problems and be closed as a result of that; a question that seems to be based on an absurd premise without reasoning can be closed as NARQ. We could institute mandatory requirements, but we also have to keep in mind we don't want to close questions just because they don't meet a long list of criteria, and adding research requirements seems to be the thin end of the wedge. However, I also get the feeling we could work on this, but I'm lacking ideas right now.

---

How can we improve?

For those of you coming in from TL;DR, here's a quick summary: We don't have a significant quality problem, and I've suggested a revised on-topic/off-topic policy. Beyond that, however, there are still improvements to be made. Based on everything above, I'm going to ask a few questions and ask the community to answer them:

1. We do have a fair number of ESL questions. The terrible ones should be (and are being) closed/deleted. However, that still leaves a fair amount. Do you think this may discourage native speakers, and if so, should we make an active effort to make sure harder questions are asked (without discouraging the ESL questions, so effectively raising the hard:easy ratio)?

2. I'm a high school student, so naturally I interact routinely with quite a large number of people who have English questions. Due to some odd logistics, I technically attend two high schools: a really tough one and a less-tough but still selective one. It's one thing to not shape the site to an audience, but quite another to go out and invite specific audiences. Should I make an effort to invite students of the first school (who I feel would work well with the site)? The second (which may produce more "proofread" questions, but still other good questions)? (I don't mean this to specifically apply to me, but from answers to this question we can get a feel for what the community wants as far as adding to itself goes)

3. In my opinion, high rep users aren't doing much question moderation, such as voting to close and delete, and much of the work falls to moderators. On an individual level, that's fine—asking and answering questions is enough of a contribution already, there's no mandate for you to also do our janitorial work. But I don't think this is a "I don't want to" problem so much as a "I don't know when to vote" problem. Do you have any suggestions as to how we can improve this?

4. Do you disagree with my conclusions about our question quality? About my on-topic/off-topic proposal? About my opinion on not mandating research except in blatant General Reference cases? Or about anything else? If so, why?

---

Here are the 15 questions I used, and brief comments. A 4 or 5 at the end indicates excellent, a 3 ok, a 2 not-so-great, and a 1 is a closed question.

• Are older senses of "anent" still alive in any dialect?—Not exceptionally brilliant (I don't believe anent, older or newer senses alike, is used at all), but a good solid question nonetheless with references and all—4
• Looking for a better term than 'benign envy' or 'mudita'—Interesting question, with lots of background—5
• Are there other words with the same weird spelling / pronunciation combo as "victual"?—A list question, which SE isn't too fond of, but research is good—2 (only because it's a list question)
• https://english.stackexchange.com/questions/34562/clarifying-that-your-intentions-are-no-other-than-constructive-closed—Closed for being far too broad; OP is reluctant to improve question—1
• Hyphenating "Evolution"—Original question was quite confusing, to the point that I misinterpreted it while editing. Once kiamlaluno and I had a crack at improving it, a ok question.—2
• https://english.stackexchange.com/questions/34553/an-representation-closed—Closed as too localized, as the question was based on an error.—1
• Why is the initial "ts" sound (e.g "tsunami") pronounced as "s"?—Good, interesting question. Was edited for clarification, but nothing significantly wrong in the first place.—4
• "Interchange" and "exchange"—A "word vs. word" question. Perhaps a little more research would have been nice, but nonetheless ok.—3
• Antonym of "dispenser"—Had research, but failed to properly communicate what meaning of "dispenser" was intended. Initial title was ambiguous. Still, not a bad question.—3
• Where did this usage of "something" originate: "I need a nap something terrible"?—A little more context might have been nice, but ok.—4
• "Application for Android" versus "application on Android"—Original question was poorly formatted, but easily understandable.—3
• https://english.stackexchange.com/questions/34532/which-tense-should-i-use-when-writing-a-post-about-a-journey-closed—Closed as off-topic—1
• Is there any difference in meaning between "She is not around" and "She is not here"?—Good, solid question—5
• Why is it "grandfather", but "great-uncle"?—Good question with research—5
• Abbreviation for "so-called"—A bit far-fetched, but understandable given the fact that one exists in German—3

Answer 1

Thanks for this. It is clear that you spent a very long time thinking about this, and the overall cohesion given its breadth is impressive, especially considering you are a high school student!

That being said, however, I would like to take up your solicitation for feedback. On your proposed on/off-topic changes, the way it is written, it seems to be implying that general questions about syntax and semantics that aren't necessarily about specific words or about whether some specific construction is grammatical are off-topic. While I agree that such questions are rare, I don't think we should give the impression that such questions are unwelcome. Here are some examples of what I am talking about:

• What is the rule for adjective order?
• How to properly write sentence with double words
• Why do we invert word order when asking a question?
• Are there any rules on the positioning adverbs should take in a sentence?
• English questions and negation with *do* in syntax

That last question, I think, is an awesome question with an awesome answer that is the kind of expert-level question that TPTB had wet dreams about when they were conceiving the Stack Exchange network. We absolutely have to not only not forbid but explicitly encourage questions like that.

So, WaiWai933, thank you for taking the time to write all this out and clarify exactly that the extent of the "problem" we have been discussing is mostly perception and very little reality.

---

As a separate issue, I just wanted to point out that I substantially defanged the unilaterally added new FAQ entry about jokes. It now is really mostly redundant and serves only as a signifier that "explain this joke to me" questions will be scrutinized more than other questions. Frankly, I don't see that questions about jokes are any different from other questions expressing uncertainty about the semantics of a particular utterance. Seeing as how there is no logically consistent reason to single them out for special treatment, I'm apt to just nix that bullet point altogether from the FAQ in a few weeks once all the Sturm und Drang has died down.

Answer 2

Do we really have a quality problem when it comes to questions submitted to EL&U?

I don't think so. One could even take the provocative view that a question cannot be stupid at all.
That's because a question is not a statement. Questions are by definition not affirmative, and only statements can be nonsensical.

For instance if I ask in EL&U.

Is the noun host (as in computer) related with the noun host in the expression "a host of people" ?

Now that is a stupid question! Of course there's no link!
Or is it?

Well no it isn't. In Middle English and in Old French an "ost" ("host") is a marching army. "L'Ost" is the army of the king. So that just as one can say "an army of people", one can also say "a host of people".
Now that the original sense of "army" has been forgotten, you even find a host of applied to things that don't march: "a host of reasons", "a host of features", and so on.

The host who gives you hospitality in his hotel deals with guests who are basically possibly hostile strangers... All these words are closely related and their mutual relations and intertwined history have been amply commented by scholars.

So that hypothetical question was not as stupid as it first looked and censoring it would actually lower the quality of the site. Although this is a made up example, EL&U's archives are full of similar examples of seemingly dummy questions bringing about interesting and unexpected answers.

It's like when your kid is entering the question age around 4 years old: he will ask loads of questions and the richness of his understanding of the world will depend on the ability of his parents to answer them correctly. Ignorant parents will just answer
  "That's the way it is and please stop asking stupid questions!".
That's actually the epitome of a stupid... answer.

So we can't have stupid questions.

• We can have poorly spelled or pourly phrased questions. They are swiftly edited by the community.
• We can have troll questions. They usually get flagged and closed quickly enough.
• We can have lazy English learner's question of little interest. These usually get closed as General Reference.

And that's it.

If we are really serious about the quality of this site, what we don't need is mods being pressured to act as ignorant parents and close apparently poor quality questions.

The quality of a question is arguably only fully appreciated once it has received a great answer.

Answer 3

Do you disagree with my conclusions about our question quality? About my on-topic/off-topic proposal? About my opinion on not mandating research except in blatant General Reference cases? Or about anything else? If so, why?

Yes, and strongly. While I think there are good questions scattered about the site, here is my impression of the newest questions:

• 4 boring interpretation/meaning of/is this sentence correct questions
• 1 closed is this sentence correct question
• 1 pun/joke interpretation question
• 1 word meaning question
• 3 comparison/differences between two words questions
• 1 interesting single-word-request
• 1 mediocre but poorly worded single-word-request
• 1 boring single-word-request
• 1 off-topic formatting question (closed)
• 1 interesting etymology question
• 1 question about metaphors

And then I got bored of doing this. So, naturally, my opinion is weighing heavily on this summary but here it is:

• 16 total
• 2 closed
• 6 boring or mediocre
• 4 "basic" questions regarding word meanings or differences
• 2 interesting questions
• 2 others

Of the 14 unclosed, about half are dead weight and I'd personally upvote a whopping 2. I am not going to link to them here because I don't really want to focus on particular questions. I just wanted to point out that your type of analysis is... well, yours. That is perfectly fine but my perception of question quality on this site is drastically different. This experiment matches up pretty well with my daily experiences.

What I really dislike about this site is that no one seems to agree on what is good or bad quality; on or off topic. Some of the questions I consider dead weight are upvoted well. Perusing the new questions page is painful for me so I've stopped doing it. I hit 10k and now I go around voting to close and upvote/downvoting stuff waiting for an interesting question.

So, yeah, I do think we have a quality problem. A bad one — one that will be kicking us in a few months. But I don't mind it so much as I mind the issue of the community mass not being able to identify the bad questions. They seem perfectly willing to upvote anything that tickles their fancy and then defend their bad questions and bad answers with long arguments in the comments.

Most of my surprise and intrigue on this site comes from seeing which questions get upvoted. The surprise and intrigue should come from interesting questions and answers. Instead, the psychology of voting is more interesting. That just seems like a bad sign.