I was reading some posts on improving question quality and narrowing the definition of the General Reference when I had an idea. What does ELU Meta think about putting a question on hold when "more research is desired" (MRD)?

The first point made here is that poorer quality questions don't show enough research. I think it's possible that when there's no research present, interesting questions can get mistaken for them -- this is what I believe happened with this question about the origin of "bunk".

Of the close-holds I know, I think there's a place for it:

  • Is the question meant for SE? (Closed as Spam)
  • Is the question meant for this SE? (Closed to Migrate, for Proofreading, etc.)
  • Is the question unambiguous? (On-hold as Unclear)
  • Is the question supported by good reasoning/research? (On-hold because more research desired)
  • Is the question sound? (On-hold because of typo or like mistake)
  • Is the question answerable from ELU's resources? (Off-topic because POB)
  • Is the question already answered from ELU's resources? (Off-topic because GR)

(I hoped to order these based on which to generally look for first, given criteria such as their dependencies on each other. But I go back and forth on the placement of some.)

I think much good can come from adopting this, particularly if we use it only when we feel the question will remain open (or become protected) when more research is added. This is tricky, because we may not fully understand a question's potential without sufficient research. (This is why I rank GR after MRD, though arguably such a question should be on-hold as unclear.) But the asker has an idea of its potential, so MRD forces the responsibility of demonstrating it back in the asker's hands instead of into those of the reviewers/answerers.

But does this responsibility ever leave the asker? Can't they edit their question on request at anytime? Why have a close-vote?

In addition to preventing mis-closure for another reason, the best practical reason I see is enforcement. This way a question must meet ELU's standards before getting an answer, if an answer has not already been supplied in comments. (Following the ranking above, the question has already been approved as right for ELU; it just needs a little more refinement is all.) If the question appears to be from homework or a test, the asker is forced to show external work to get an answer from ELU. It sounds a little harsh said that way, but I believe it is also reasonable and fair -- provided we have members willing to go through recently edited MRD questions to reopen them as needed.

From the list of the first point made above, the asker who gets their question reopened by necessity amends their question's following shortcomings:

  • Showing little research or effort
  • Being extremely short
  • Not being revised

The remaining points may end up amended in the process too.

I tried to choose the simplest but most general wording I could, but it's possible there's a better title. I have toyed with replacing Research by Detail/Effort, since the former could tempt us into looking at the number of links/quotes per question. I myself have posted questions that presented little (formal) research because I recognized a pattern but did not know where/how to search for its description. But a downside to using Detail/Effort is that the asker may add more informal discussion when formality is desired. I think as long as we flaggers agree that Research can mean the user's own reasoning / readiness to provide examples / etc., we'll be ok.

I envision the MRD pane leading to a link of authoritative resources that might help the question. As a parallel to GR's help text:

You can improve this question and have it reopened by the community by adding citations from other sources. A list of accepted resources can be found here: LINK

Where LINK leads to a page mapping topics to handy websites:

  • Origin? Try Etymonline / Google nGrams!
  • Meaning? Try MW or other dictionary!
  • Grammar? ...

What do you think, ELU Meta?

  • 1
    I like "research" because it implies effort by the asker. If we wanted it to sound less daunting, maybe "information" would do. – Kit Z. Fox Jul 1 '14 at 11:00
  • Isn't this what "It's unclear what you're asking" was intended to cover? Maybe we can usurp that reason. – MrHen Jul 1 '14 at 14:31
  • 4
    @MrHen To me, Unclear What You're Asking is saying "you need to edit your question for clarity", not "add more background". Very different action needed. – Avner Shahar-Kashtan Jul 1 '14 at 15:35
  • "Closed as Spam"? you mean "flagged as spam"? – Braiam Jul 2 '14 at 16:53
  • Please see this related question and its answers. – tchrist Jul 2 '14 at 16:58
  • 1
    I enjoyed the discussion of the "Lacks Minimal Understanding" flag at the link you provided, tchrist—thanks! It occurs to me that it might make sense to permit closing bad questions for the neutrally expressed (and very accurate) reason: "Answered in Comments." This might also help with the unanswered questions queue, where people typically aren't inclined to vote up a simple (but dispositive) answer to a dumb question, especially when someone has already answered it in a comment, so the question lingers in the queue and annoys MrHen (and me). What do you all think? – Sven Yargs Jul 2 '14 at 21:42
  • @tchrist I would like to see MRD as more open/forgiving than the stance presented in the "Reviewing Awful Questions" link. For this flag to meet its full potential there must be pardons; I picture them being loose at first. A question is flagged as MRD, its user adds more research (a relevant link or two from the list), and the question is reopened. Rather than stamping the question as awful or the user as incapable/unwilling/moronic, I'd like MRD to help new users get in the habit of showing their work, of asking an informed (when not otherwise interesting) question. – user39720 Jul 2 '14 at 23:23
  • Braiam: I think I might. I get the terminology wrong sometimes. In my comment below Sven's answer, for instance, I accidentally said MRD was a close-reason when it's crucial for MRD to be an on-hold reason as stated in the title. @SvenYargs I've seen many questions answered that way, so I think that's a good strategy. (It reduces redundancy for one, because typically the accepted answer reiterates the comment.) If there were a way to ensure that it answered the user's question (e.g., it's used when the asker verifies it answered his question), I'd say it's the best strategy I've heard! – user39720 Jul 2 '14 at 23:42
  • 1
    @dingo_dan “On-hold Reason” ≣ “Close Reason” – tchrist Jul 2 '14 at 23:51
  • See, folks, that's how bad I am with these terms! :) In my experience, some questions on ELU are automatically closed if vote quota met instead of put on-hold for 5 days as suggested here? If the two terms are indistinct here, I would prefer MRD to be marked as on-hold (doesn't matter to me if for 5 days or permanently), because I feel it has an element of temporariness/reclamation that closed (even though there's a reopen option) doesn't always have. But it's no biggie, since I think the close-vote's help dialog should be enough to convey that already. – user39720 Jul 3 '14 at 0:19
  • To be a bit pedantic it's not really that more research is desired, many people do some research they just don't include the results of that research in their question. It's more of a "Please include the results of your research and why you found it unsatisfactory" – Jim Jul 4 '14 at 1:36
  • @Jim I consider MRD to be like Unclear, being about the question as posted. An Unclear tag is not saying that the person does not have a clear question in mind, but that the question presented is not clear; MRD is not saying the person did not do the research, but that it is not currently presented, which may lead to the rehashing noted in Sven's answer. (But there is room to clear this ambiguity not only in the title but also in the help text box.) If research results are posted and it's unknown why they're unsatisfactory, the question sounds like it's Unclear. – user39720 Jul 4 '14 at 1:57
  • @dingo_dan- yes, I get that, my point was that the actual words "MRD" don't quite say that and that while I am fully supportive of the intent behind your suggestion, I think better words are needed to convey that intent. In other words, I like the intent of MRD, I just think it should be different letters. – Jim Jul 4 '14 at 4:57
  • @Jim I can see how "More Research Results Desired," which I feel addresses the concern raised, emphasizes that the info (not the process) is omitted. However I also feel it's slightly redundant, since that's what the suggested help text is saying. Similarly, "More Research Desired in Post(ed Question)" feels as redundant as "(Post[ed Question]) Unclear." I am not fully satisfied with MRD's title now either and not sure why yet. (I think it's the rigor/formality implied by research.) Boy, it's hard work coming up with a good title but it's also fun to think about! :) – user39720 Jul 4 '14 at 15:34
  • 2
    "This question is off-topic because it was answered in the comments" is probably the least satisfying close reason I've seen proposed. – snailboat Jul 4 '14 at 17:45

I think that's a great idea; as an additional note, I see that the hint text by the question upvote arrow on all three sites and metas I'm usually on says "This question shows research effort; it is useful and clear". I think that having, in effect, a formal request for more research would (or at least could) result in better questions. And I'll take a "could". I'll even take a "might".


One of the most popular questions of the past week was "Soccer mom": why soccer?. The accompanying commentary made clear that the OP understood what the term meant and when it had emerged, but wanted to know why its coiner had chosen that particular sport. It was a great question, and a number of people tried to answer it.

But in a formal, show-your-work sense, the question wasn't especially well researched: The "soccer mom" questioner gave no evidence of having done any independent research in an effort to answer the question himself. He simply described how much he did know on the subject, and asked EL&U readers to inform him about the things he didn't know.

To me, his handling of the question was exactly appropriate to the situation—and it was very much the same approach that (it seems to me) Yoichi Oishi uses in formulating his questions, which many (though not all) of us here consider models of their kind.

First and foremost, a good question asks something that people who care about language recognize as interesting (though it may take some research to discover just how interesting it is). Second, the question summarizes what the questioner already knows, so answerers don't waste time rehashing stuff that isn't in doubt. And third, the question is clear enough about what the questioner wants to know that answerers don't waste time guessing at what it might be or pursuing false objectives.

But satisfying these criteria doesn't inherently require any research; it simply requires having a good question to begin with. To the extent that a lack of research constitutes a valid objection to a question, I think, it does so because such research would have disposed of the question; that is, a modest amount of preliminary research by the questioner would have revealed the answer to the questioner without requiring the involvement of EL&U in the process. However, that logic amounts to saying that a modest amount of preliminary research would have revealed that the question itself wasn't good (by the first criterion I mentioned above) in the first place.

Here's where things get problematic for me. More than once I've looked into a seemingly simple question and found that the stock answers offered by general references either disagree in certain particulars or overlook important evidence that complicates the question (and answer). If we insist that questioners show that they have undertaken some independent research before coming to us for answers, those questioners will almost certainly encounter a stock answer from a general reference and will consider the matter closed. At times, this result isn't good for the cause of truth.

I understand that EL&U suffers from a serious problem of being inundated by tides of trivial and boring questions. But if "put on hold until you demonstrate that you've done some elementary research on your own" is actually designed to weed out bad questions by demanding more effort of the people who ask them than we do of the people who ask good questions, it establishes an unacknowledged double standard. Again, bad questions aren't bad because they aren't adequately researched; they're bad because there's nothing to them.

Most crucially, to the extent that seemingly bad questions may occasionally be good questions in disguise, I think that sending the questioner back to do more research is actually counterproductive because such a questioner will rarely go deeply enough into the research to find the hidden complications that make both the question and the answer(s) interesting. We're the ones with the multiple reference works on hand and the sense of what to look for in the way of inconsistencies, ambiguities, and other complications.

Perhaps the most honest explanation—"put on hold as being a question of no serious interest to anyone but the questioner"—sounds too rude or dismissive to pass muster. But the farther the reason we give for putting a question on hold is from the real reason for our doing so, the likelier we are to reap unintended and undesirable consequences from our policy.

Ultimately, I don't know how we can filter out all of the bad questions and keep all of the good ones; but even at the cost of tolerating bad ones an hour or two longer than we otherwise might, I would like to see a bit more patience in our handling of certain types of seemingly obvious (and therefore seemingly bad) questions, in the hope that research by an interested answerer might discover something valuable in it.

I hope I don't sound too negative about dingo_dan's proposal, because it's clear to me that he recognizes the desirability of keeping "hidden good" questions in play while disposing of tons of slag. I just want to make sure that we don't put too high a value on research as a necessary characteristic of good questions, or as a hoped-for way to transform bad questions into good ones or to make bad ones simply disappear.

  • The “put on hold as being a question of no serious interest to anyone but the questioner” is what Too Localized was all about. – tchrist Jul 2 '14 at 0:27
  • Thank you for the thoughtful response! I agree on unintended consequences. My biggest concern is keeping this flag about asker effort. Now, answerers are drawn to held/closed mostly by whim. Ideally a member (OP?) would see MRD, reason "It's not Unclear, so it must make sense; it's not GR/Dup, so it may not be answered already;...", use it to find questions ripe for research and appeal on Meta if good one found. (Maybe titles end with [closed:MRD] for the gist at a glimpse.) But how do we maintain such a consistency? Delayed flagging is better, because it does not make OP feel hastily judged. – user39720 Jul 2 '14 at 0:54
  • 2
    I don't think "He ... described how much he did know on the subject" is (in this instance) significantly different from "He described the results of his research on the subject". – Matt Gutting Jul 4 '14 at 18:58
  • @MattGutting I quite agree. Giving a clear explanation of how far your own accumulated knowledge (whether inherent or from research into this specific question) will take you is tantamount to sharing your research. Whether either counts as proper research to try to answer the question is a different matter. If the question is simple etymology of a common word, stating that you don’t know where it comes from won’t cut it; but then, nor (at least with me) will saying that you’ve looked in one dictionary and it didn’t have the etymology, even though that’s ‘research’. – Janus Bahs Jacquet Jul 10 '14 at 16:27

You must log in to answer this question.