26

Table of Contents

This is a really, really, long post. And yes, I hate to create another post on our current "quality" situation, but this post is seriously long to the point where I don't want to distract from anyone else's post. In addition, this represents my personal view only.

In fact, it's so long that I'm including a table of contents to help you read only the sections you want to.

  • Previous Questions
  • Audience, and why looking at is isn't entirely productive
  • Do we have a quality problem?
  • What is Jeff concerned about?
  • Close reasons
  • What is off-topic?
  • Question Pre-requisites
  • How can we improve? <-- This is the TL;DR one to read

Previous Questions

First things first—I'm going to quickly link to the questions here on Meta.English about how we can improve on the recent situation:

One is on whether we have a problem:

Three are on what type of audience is appropriate:

And another is an attempt to clarify the on-topic/off-topic policy:


Audience, and why looking at it isn't entirely productive

I think the best way to solve this problem successfully is to take a close look at question quality and close reasons. However, before I do that, I want to address why I don't think questions about our audience are that helpful.

If we take a look at SE in general, none try to scope for audience. Instead, the scope is for on-topic and off-topic, and both beginner questions and expert questions are welcome. In fact, the majority of the answers to those two earlier questions make it clear that this is how most of our users feel—if you have the ability to consult a dictionary and/or thesaurus before posting, your question is probably welcome. In other words, anyone who genuinely wants to learn is welcome—a philosophy that pretty much applies to any SE site. So while it's good that the community is reminded of this, there's a limit to how much use it can be to us.

With that, I'm going to try not to mention audience again. (Ok, actually I do mention it one more time.)


Do we have a quality problem?

Let's see if there is a problem. I know this has been discussed, but I'm going to take a non-holistic, methodical approach.

I looked at the last 15 questions asked as of writing, and here's a quick summary as they were originally posted. The details for the questions are at the bottom of this post.

  • 6 excellent questions (of which 3 I could not fault, and the other 3 were nitpicking problems)
  • 4 ok questions
  • 2 not-so-great question
  • 3 closed, none as exact duplicate

In other words, 10 were fine, and 5 weren't. Of the bad 5, 1 was edited and is now equal to a ok question, 1 was a list question, 1 was based on an erroneous assumption, 1 was provided without enough detail, and the last 1 was off-topic.

How is that? That number means that 2/3 of all questions posted, examined before editing, are fine—and that the remaining 1/3 isn't. I've had a quick chat with mods from a few other released sites, and they're also running at about that amount. In fact, that's a solid number, and shows that there isn't a rampant problem.


What is Jeff concerned about?

Let's take a quick look at Jeff's concerns (And you're free to edit any more you have into this post, Jeff):

  • Questions about jokes
    • Jeff thinks that they deteriorate site quality. The community disagrees, I disagree, but Jeff isn't budging. I realize this has caused this entire wave of debate—it's definitely caused several of our members to leave or consider leaving.
    • Ideally, I'd love to see Jeff come around to our way of thinking. However, assuming that that doesn't happen, the compromise is that we allow Jeff this one concession in return for having built Stack Exchange. I feel and hope that the community will allow this, in return for not allowing this sort of drastic action to happen again.
    • How can we stop this from happening again? I suggest a set of clear, objective criteria that we can use to determine on-topic and off-topic—a series of questions/flowchart of sorts. I've expanded this below. If we can get Jeff to agree to also use this flowchart so that it's entirely objective, I doubt it'll ever happen again.
  • Questions about things of a perhaps less-than tasteful manner
    • More specifically, the titles need to be clean. As I understand it, EL&U seems to be sanitizing these titles as quickly as it sees them. Jeff wants them even cleaner, but we don't want the titles to be overly vague. I can only suggest we keep the importance of censoring titles in mind when dealing with these sort of questions.

Close reasons

I'd just quickly like to go over these:

  • exact duplicate
  • off topic—should be determined by a set of clear, objective criteria, a proposal of which is set out below
  • not constructive—this is the old S&A close reason—one question, one answer (certain list questions of exceptional quality excepted)—otherwise it gets closed
  • not a real question

    • if a complete answer to the question is as long as this post because of how broad the question is, something is seriously wrong—in other words, don't be vague and dance around what you're trying to get at
    • in addition, if the question is so basic that it demonstrates a lack of understanding of basic English, such as SVO order, then it's NARQ—does not apply if asking how or why a particular basic form of English came about (this, by definition, only applies to ESL questions)
  • too localized—it's also not a real question, but this time because it's far too specific (which, by the way, is not the same thing as detail, which is good)
  • general reference—if I consult Wiktionary, Dictionary.com, Thesaurus.com, Google, or Wikipedia (or other appropriate well-known sources), and find the answer just sitting there without any interpretation required and the answer is utterly boring, it's general reference

What is off-topic?

In light of the recent debate about questions about jokes, here's a proposal for revising on-topic and off-topic:

This only determines whether a question is on-topic or off-topic. They can still be closed for any other close reason.

  • Is your question about English words (e.g. what does a word mean or "word1 vs word2") or phrases (including idioms and the like)? On-topic (covers the existing usage, word choice, etymology, pronunciation, and spelling on-topic reasons)

  • Is your question about grammar or syntax, and if so, does it make specific references to what is of concern? On-topic (covers the existing grammar and "problems encountered..." on-topic reasons, while excluding proofreading)

  • Is your question about how English varies? On-topic (covers the existing etymology and dialect differences on-topic reason)

  • Is your question about punctuation as it relates to English? On-topic (covers, well, duh)

Otherwise, it's off-topic. This does its best to cover the existing /faq on-topic and off-topic things, but improvements are no doubt possible.


Question Pre-requisites

I'd like to address one final thing, and that's what needs to go into a question before you should post it. Of the 12 open questions, 6 had research, and 6 lacked it. I don't think that that's a particularly bad number—if you look at the questions themselves, some can't be easily researched, which brings it down to 2 that could actually use research.

However, the question remains of how we address it. I don't see much to do—a question without research where research could be easily done is very likely to have other problems and be closed as a result of that; a question that seems to be based on an absurd premise without reasoning can be closed as NARQ. We could institute mandatory requirements, but we also have to keep in mind we don't want to close questions just because they don't meet a long list of criteria, and adding research requirements seems to be the thin end of the wedge. However, I also get the feeling we could work on this, but I'm lacking ideas right now.


How can we improve?

For those of you coming in from TL;DR, here's a quick summary: We don't have a significant quality problem, and I've suggested a revised on-topic/off-topic policy. Beyond that, however, there are still improvements to be made. Based on everything above, I'm going to ask a few questions and ask the community to answer them:

  1. We do have a fair number of ESL questions. The terrible ones should be (and are being) closed/deleted. However, that still leaves a fair amount. Do you think this may discourage native speakers, and if so, should we make an active effort to make sure harder questions are asked (without discouraging the ESL questions, so effectively raising the hard:easy ratio)?

  2. I'm a high school student, so naturally I interact routinely with quite a large number of people who have English questions. Due to some odd logistics, I technically attend two high schools: a really tough one and a less-tough but still selective one. It's one thing to not shape the site to an audience, but quite another to go out and invite specific audiences. Should I make an effort to invite students of the first school (who I feel would work well with the site)? The second (which may produce more "proofread" questions, but still other good questions)? (I don't mean this to specifically apply to me, but from answers to this question we can get a feel for what the community wants as far as adding to itself goes)

  3. In my opinion, high rep users aren't doing much question moderation, such as voting to close and delete, and much of the work falls to moderators. On an individual level, that's fine—asking and answering questions is enough of a contribution already, there's no mandate for you to also do our janitorial work. But I don't think this is a "I don't want to" problem so much as a "I don't know when to vote" problem. Do you have any suggestions as to how we can improve this?

  4. Do you disagree with my conclusions about our question quality? About my on-topic/off-topic proposal? About my opinion on not mandating research except in blatant General Reference cases? Or about anything else? If so, why?


Here are the 15 questions I used, and brief comments. A 4 or 5 at the end indicates excellent, a 3 ok, a 2 not-so-great, and a 1 is a closed question.

  • 7
    'I'm going to take a scientific ... approach' Based on what comes after that: please, please don't enter into science as a profession. – Grant Thomas Jul 18 '11 at 11:45
  • Yes, definitely a poor choice of words. I've revised that bit. – waiwai933 Jul 18 '11 at 16:02
  • 2
    As a newcomer - where are jeff's original concerns? (or is this really only meant for people who were here prior to discuss and new people shouldn't know what happened since you all are focusing on the future?) (Were people asking artful dick jokes?) – bmike Jul 18 '11 at 16:30
  • @bmike Here's the one about jokes—the other one has been deleted by another mod. – waiwai933 Jul 18 '11 at 16:33
  • Wow - that link makes perfect sense and fills in the context perfectly. What a laudable goal to set such high standards to keep this very librarian, research and serious. I wish you all success in carving out a specific, focused niche here. – bmike Jul 18 '11 at 16:44
  • @waiwai933: I'd like to ask something but I'm not sure I should open another topic, so I'll ask here: What is the priority when closing questions? I mean, the close-reasons priority chart, let's say... (If this deserves a Meta discussion, tell me, I'll open it.) – Alenanno Jul 18 '11 at 19:37
  • @Alenanno Do you mean what reason to choose when a question is faulty in multiple areas? I'd suggest Duplicate, then NC, since those are usually reviewed before deletion, then NARQ/TL/GR/OT (in any order, since it doesn't matter), then OT with migration, since we don't want to, as Jeff so eloquently puts it, "migrate crap". – waiwai933 Jul 18 '11 at 21:27
  • Long, but worth it, I think. Certainly diligent,perspicacious,and eloquent, thank you. Personally I'll be glad to hear no more of the [target] audience. Qs about jokes should handled carefully, if at all; I'm with Jeff on that one. More people should take it upon themselves to downvote (or to close) according to what they don't like/care about, not shelter anonymously behind "broad church" aspirations. That would do much to clarify what kind of questions are welcome, to be reflected by FAQ, not dictated. – FumbleFingers Jul 19 '11 at 1:31
  • 1
    @FumbleFingers The FAQ can't be completely descriptive. To some extent, it must be prescriptive, because human nature causes us to encourage "fun" questions that increasingly border on being off-topic, to the point where our scope widens significantly. If the scope of a Q&A site becomes too broad (such as Y!A), the quality of answers can't be maintained. So while I agree that the community at large needs to take a bigger role in determining on-topic/off-topic, the direction should be primarily Meta discussion -> FAQ (but yes, downvoting and closing do contribute a lot). – waiwai933 Jul 19 '11 at 6:23
  • As you obviously realise, FAQ a complex but crucial issue. I've raised my point as a specific Question – FumbleFingers Jul 19 '11 at 13:40
  • 3
    to be clear on the jokes part, I never said that all joke questions must be closed. I meant that they must be held to an even higher standard because people love jokes and are willing to cut corners on quality -- intentionally or not -- for the sake of being amused. – Jeff Atwood Jul 20 '11 at 10:17
14

Thanks for this. It is clear that you spent a very long time thinking about this, and the overall cohesion given its breadth is impressive, especially considering you are a high school student!

That being said, however, I would like to take up your solicitation for feedback. On your proposed on/off-topic changes, the way it is written, it seems to be implying that general questions about syntax and semantics that aren't necessarily about specific words or about whether some specific construction is grammatical are off-topic. While I agree that such questions are rare, I don't think we should give the impression that such questions are unwelcome. Here are some examples of what I am talking about:

That last question, I think, is an awesome question with an awesome answer that is the kind of expert-level question that TPTB had wet dreams about when they were conceiving the Stack Exchange network. We absolutely have to not only not forbid but explicitly encourage questions like that.

So, WaiWai933, thank you for taking the time to write all this out and clarify exactly that the extent of the "problem" we have been discussing is mostly perception and very little reality.


As a separate issue, I just wanted to point out that I substantially defanged the unilaterally added new FAQ entry about jokes. It now is really mostly redundant and serves only as a signifier that "explain this joke to me" questions will be scrutinized more than other questions. Frankly, I don't see that questions about jokes are any different from other questions expressing uncertainty about the semantics of a particular utterance. Seeing as how there is no logically consistent reason to single them out for special treatment, I'm apt to just nix that bullet point altogether from the FAQ in a few weeks once all the Sturm und Drang has died down.

  • I've edited the on-topic section and hopefully addressed those syntax questions. I didn't mean to exclude them; I just happened to be working from the existing faq in writing those and didn't realize I had inadvertently left them out while trying to keep proofreading out. Let me know if the change doesn't suffice (or edit it yourself). As for the joke questions, I'm glad to hear that—I wasn't sure from the existing questions how that had turned out. – waiwai933 Jul 18 '11 at 5:56
  • @waiwai seems OK now, but I'm not convinced it's good enough to replace the existing on/off-topic FAQ list. Maybe as a supplement, or alternative formulation. Or maybe we just need some concrete examples in the FAQ. – nohat Jul 18 '11 at 19:46
  • 1
    I strongly encourage editing to make the "almost there" questions suitable. It never ceases to amaze me that users will take the time to compose incredibly detailed, excellent answers -- but inexplicably leave the mediocre-at-best question completely untouched. Protip: if you want the exposure you deserve for writing an excellent answer, give the question a little editing TLC too! Nobody is going to click through when the question title and body are subpar... – Jeff Atwood Jul 20 '11 at 10:20
11

Do we really have a quality problem when it comes to questions submitted to EL&U?

I don't think so. One could even take the provocative view that a question cannot be stupid at all.
That's because a question is not a statement. Questions are by definition not affirmative, and only statements can be nonsensical.

For instance if I ask in EL&U.

Is the noun host (as in computer) related with the noun host in the expression "a host of people" ?

Now that is a stupid question! Of course there's no link!
Or is it?

Well no it isn't. In Middle English and in Old French an "ost" ("host") is a marching army. "L'Ost" is the army of the king. So that just as one can say "an army of people", one can also say "a host of people".
Now that the original sense of "army" has been forgotten, you even find a host of applied to things that don't march: "a host of reasons", "a host of features", and so on.

The host who gives you hospitality in his hotel deals with guests who are basically possibly hostile strangers... All these words are closely related and their mutual relations and intertwined history have been amply commented by scholars.

So that hypothetical question was not as stupid as it first looked and censoring it would actually lower the quality of the site. Although this is a made up example, EL&U's archives are full of similar examples of seemingly dummy questions bringing about interesting and unexpected answers.

It's like when your kid is entering the question age around 4 years old: he will ask loads of questions and the richness of his understanding of the world will depend on the ability of his parents to answer them correctly. Ignorant parents will just answer
  "That's the way it is and please stop asking stupid questions!".
That's actually the epitome of a stupid... answer.

So we can't have stupid questions.

  • We can have poorly spelled or pourly phrased questions. They are swiftly edited by the community.
  • We can have troll questions. They usually get flagged and closed quickly enough.
  • We can have lazy English learner's question of little interest. These usually get closed as General Reference.

And that's it.

If we are really serious about the quality of this site, what we don't need is mods being pressured to act as ignorant parents and close apparently poor quality questions.

The quality of a question is arguably only fully appreciated once it has received a great answer.

  • I suppose it would be bad form to actually ask your question, answer it myself, and wait for the upvotes to come rolling in! Seriously, I agree almost completely. What we need is better Answers more than better Questions. And more people upvoting anything that's really "better", as opposed to simply being "true". It would be nice to encourage better stuff without getting fanatical about clearing out not-quite-so-good stuff. – FumbleFingers Jul 20 '11 at 3:32
  • 1
    I actually toyed with the idea to really submit the host of question for real - just to see if he'd be really closed as "low quality". I only shied away because I suspect it would just have attracted more pity than criticism (as in "poor Alain is really weird at times"). – Alain Pannetier Φ Jul 20 '11 at 19:17
  • I'd have thought you already gave the game away about weirdness with your unseemly haste to help @Malovolio mock a French friend lol – FumbleFingers Jul 20 '11 at 20:15
  • To be honest, I think no one can come anywhere close to the Brits when it comes to self-derision... – Alain Pannetier Φ Jul 20 '11 at 20:43
  • Ah! Les rostbiffs! Where would the French be without us? They'd have to insult the Krauts instead, and prolly get invaded again! :) – FumbleFingers Jul 20 '11 at 21:05
2

Do you disagree with my conclusions about our question quality? About my on-topic/off-topic proposal? About my opinion on not mandating research except in blatant General Reference cases? Or about anything else? If so, why?

Yes, and strongly. While I think there are good questions scattered about the site, here is my impression of the newest questions:

  • 4 boring interpretation/meaning of/is this sentence correct questions
  • 1 closed is this sentence correct question
  • 1 pun/joke interpretation question
  • 1 word meaning question
  • 3 comparison/differences between two words questions
  • 1 interesting single-word-request
  • 1 mediocre but poorly worded single-word-request
  • 1 boring single-word-request
  • 1 off-topic formatting question (closed)
  • 1 interesting etymology question
  • 1 question about metaphors

And then I got bored of doing this. So, naturally, my opinion is weighing heavily on this summary but here it is:

  • 16 total
  • 2 closed
  • 6 boring or mediocre
  • 4 "basic" questions regarding word meanings or differences
  • 2 interesting questions
  • 2 others

Of the 14 unclosed, about half are dead weight and I'd personally upvote a whopping 2. I am not going to link to them here because I don't really want to focus on particular questions. I just wanted to point out that your type of analysis is... well, yours. That is perfectly fine but my perception of question quality on this site is drastically different. This experiment matches up pretty well with my daily experiences.

What I really dislike about this site is that no one seems to agree on what is good or bad quality; on or off topic. Some of the questions I consider dead weight are upvoted well. Perusing the new questions page is painful for me so I've stopped doing it. I hit 10k and now I go around voting to close and upvote/downvoting stuff waiting for an interesting question.

So, yeah, I do think we have a quality problem. A bad one — one that will be kicking us in a few months. But I don't mind it so much as I mind the issue of the community mass not being able to identify the bad questions. They seem perfectly willing to upvote anything that tickles their fancy and then defend their bad questions and bad answers with long arguments in the comments.

Most of my surprise and intrigue on this site comes from seeing which questions get upvoted. The surprise and intrigue should come from interesting questions and answers. Instead, the psychology of voting is more interesting. That just seems like a bad sign.

  • 1
    Thanks for doing your own analysis! I think your results also indicate we don't have a question quality problem—most of the questions on SE sites are boring ones; the interesting ones are the ones we want, but if we only allowed those, we wouldn't have enough questions to keep the site going (although I will admit, 4 def./word diff. questions does seem a bit too much). Also, there's nothing I or the community can really do about those, except close general reference. – waiwai933 Jul 27 '11 at 21:34
  • 1
    However, I do think your points are indicative of a voting problem, but then again, "interesting" is subjective. I'll definitely think about how we can improve this, but if you have any suggestions, please feel free to propose them. – waiwai933 Jul 27 '11 at 21:34
  • @wai: I think closing as General Reference or Too Narrow is a great solution. I tend to look at quality from the perspective of, "Do we have good quality questions?" not "Do we have bad quality questions?" I would rather answer yes to the former than no to the latter. In that sense, 2 of 14 seems unfortunate. But even taking the other approach, I would have been happy closing what I referred to as dead weight. – MrHen Jul 27 '11 at 21:40

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .