7

Part of understanding the impact of three-vote closure is to understand what impact it's having on y'all. I'd like to ask anyone who feels up to it to write an answer to this question and to tell us about your experience over the past few weeks.

The sorts of things I'm interested in knowing (feel free to address all or none or add your own) -

  • Did this help?
  • Did it hurt?
  • Did it make you more interested in close/reopen reviewing? Less?
    • Are there things other than 3-vote close impacting your interest in reviewing?
  • If you can't vote but do flag, did this make you more/less likely to flag?
  • Would you prefer to go back to 5?
  • Should it be permanent?
  • What would you like me to look into when it comes to analyzing the data from the test?
  • Were there any side-effects (good or bad) that you want me to be aware of?

These are merely prompts, so feel free to add anything I'm missing that you think is worth saying. This is focused on how you're feeling about it and what data you want me to look at, so don't feel like you need to analyze the data over the last 30 days - though you're welcome to if that's how you figure out what you feel.

If you have any outstanding questions, also feel welcome to ask them here.

The test will run until about Monday the 21st of June, at which point we'll be resetting the votes to close up to 5 while I review the responses here and dig into the data that we've been collecting.

9
  • Here’s a high-level take I posted under your original Q on my first impressions: english.meta.stackexchange.com/a/14991/55623 . – Dan Bron Jun 4 at 19:44
  • 1
    I appreciate it! :) – Catija Jun 4 at 19:45
  • One thing I think should be considered is if a question was closed by 5 privileged users under the old rules, should it (not) be eligible for re-opening by only 3 privileged users under the new rule? Context for this idea is [this recent meta post] about just such a case; I’m contra re-opening but it’s moot since it already happened, despite 3 people (besides myself) who upvoted my contra answer: english.meta.stackexchange.com/q/15005/55623 . – Dan Bron Jun 4 at 19:45
  • That's an interesting question - my immediate feeling is that the simplest solution is to have all questions require only three either way, regardless of how many votes were needed to close in the first place and that's how it works currently. To make it so that questions closed with five require five to reopen, we'd have to build something special, which is out of scope for this project. – Catija Jun 4 at 19:47
  • 2
    Fair enough. Developer time is precious and I myself would vote to apply it to other things first. – Dan Bron Jun 4 at 19:48
  • 2
    So far I've been pleasantly impressed. The chaff is getting winnowed out sooner, and that's good. It has motivated me to vote to close more often, but it hasn't changed my reviewing or flagging habits any so far. It does mean I can find interesting questions more easily, and I like that. On the whole, I'd like it to become permanent. – John Lawler Jun 4 at 21:23
  • In the direction of an answer... I only notice what I notice... I've been more active on reopen questions. I don't notice the junky questions that probably -should- be closed and I certainly don't vote to reopen most closed things. But I think that's a good balance, more things closed but then checking a lottle more to reopen things just in case. – Mitch Jun 4 at 21:53
  • 2
    My impression (I have not monitored myself quantitatively) is that when I vote to close, the question usually ends up closed. This gives me some confidence in my judgement. Consequently I like the 3 votes system because it gets the job done quicker. – Anton Jun 6 at 22:13
  • The problem with the 3 votes closing system is that there is a herd of eager closevoters out there who will always find a “good” reason to closevote whatever question comes at hand. The same is true for those who will eagerly reopen those questions. The result is an increasing activity in closing and reopening … is that good thing for the site? – user 66974 Jun 9 at 6:51
8
  1. Did this help?

    Yes, unequivocally. As John Lawler as commented, "the chaff is getting winnowed out sooner, and that's good."


  1. Did it hurt?

    No. It's only done good for the site. It's possible that some new contributors' questions may seem to have been treated harshly, but getting "Not here" or "Not enough information" feedback more quickly might actually be kinder than having a poor question hang around.


  1. Did it make you more interested in close/reopen reviewing? Less?

  2. Are there things other than 3-vote close impacting your interest in reviewing?

  3. If you can't vote but do flag, did this make you more/less likely to flag?

    A diamond makes these rather difficult to answer.


  1. Would you prefer to go back to 5?

    Absolutely not.


  1. Should it be permanent?

    Yes, definitely.

    In fact it should be left at three until your review decides that three is A Bad Thing and the number should change (and I'm confident it won't find that). That would be akin to withdrawing a drug from patients on a clinical trial in order to assess the results, even though the treatment is immediately seen to be massively effective and the patients would die without it. Approving the drug after that will treat the disease, but there will be casualties in the meantime.


  1. What would you like me to look into when it comes to analyzing the data from the test?

    Since only three votes are required to re-open, it's easier to re-open questions which were over-zealously closed. It would be worth investigating the number of three-vote closures which three other people thought were wrong and re-opened.

    It might be worth seeing the number of five-vote closures which got to three but didn't make it to five: these are questions which probably should be closed but five votes was too onerous to accumulate. You could compare that to the number of three-vote questions with aged-away votes.

    Another metric might be the number of closures or re-opens where a moderator voted before the final vote, hastening the process. I suspect that both have happened less often under the three-vote regime.

    It would be interesting to see how many five-vote closures were re-opened with three. I would suggest that this is a contra-indication, and a lot of these means it's too easy to re-open.

    I'm not a statistician, but these strike me as measures which might bear some examination (and probably some tweaking to provide useful figures). And I note that Mitch has commented that his review habits have changed, so the conditions are not the same for direct comparison.


  1. Were there any side-effects (good or bad) that you want me to be aware of?

    One side-effect is that the number of re-open votes needed is also reduced. That could be a bad thing. It might be a good thing. Which it is could be determined by the re-open analysis.


On the whole, subjectively it’s a huge improvement which I’d like to stay.

1
  • 2
    100% enthusiastically endorsed. Thanks for investing the time and effort of posting an answer. I am pleased it carries the additional weight of coming from a fancy, diamond-encrusted moderator. – Dan Bron Jun 4 at 22:44
7

I think of the stream of questions posted on English Language & Usage as falling into four categories: uninteresting off-topic questions, uninteresting on-topic questions, interesting off-topic questions, and interesting on-topic questions.

The site benefits tremendously from removing the uninteresting off-topic questions as quickly as it can—and reducing the close votes required to do so from five to three certainly serves that purpose.

The second-largest category of questions posted on EL&U are uninteresting on-topic questions—duplicates and near-duplicates of the same 250 "grammar" issues that question posters have asked about here for the past decade, single-word requests of marginal utility, etc. Review-queue participants will undoubtedly disagree about whether a particular question is interesting or not, but I doubt that many question answerers feel much disappointment at the prospect that an arguably on-topic question of little interest is more likely to be closed today than it would have been in the past.

The number of interesting questions posted on EL&U on a given day is usually quite small, and some of those questions are disqualifyingly off-topic. For example, MrHen, an early stalwart at EL&U, asked the on-topic question, Where did "duck, duck, gray duck" come from? back in 2013, and its suitability for this site has (as far as I know) never been doubted. But the question When did “Duck, Duck, Goose“ migrate from Sweden to America?—although it might be reworked as an on-topic question about the English language (if, say, it asked when the phrase "duck, duck, goose" as the name for a children's game first appeared in North America)—is, as posed, properly a History Stack Exchange question, not an EL&U question.

Interesting on-topic questions may be the rarest category of all—and unfortunately their suitability for EL&U isn't always immediately evident. For example, the interesting on-topic question The phrase "do the lions", asked four days ago, quickly drew two close votes (out of the three needed to close it and prevent further answers from being posted) before the historically proverbial aspect of the expression—which the question poster had addressed in the original post—received any serious attention. This isn't to criticize the two close voters in this case: they had probably never encountered "doing the lions" as a proverbial expression (I never had either; it seems to have died out sometime in the early twentieth century) and so were inclined to see the question as a general-reference query about geography and the verb "do." But the very fact that these well-informed site participants had never heard of a once-commonplace proverbial phrase is evidence in favor of the posted question's worthiness. If Andrew Leach and I had not independently inquired into the history of usage of "doing the lions," the question would almost certainly have been closed as an off-topic irrelevancy.

A larger number of interesting, on-topic questions have this quality than you might suppose. And to the extent that close voters vote on the basis of what they are familiar with, without checking to see whether a particular question asks about a past or present meaning or usage that they are simply ignorant of, they put these questions in jeopardy. Shifting from five-vote closure to three-vote closure significantly narrows the widow of opportunity that an answerer has to research and post an answer to such a question without having to post a question/argument on Meta advocating that site participants reopen it.

I have argued elsewhere that interesting, on-topic questions already tended to be closed too often and too quickly under the five-vote closure system; moving to a three-vote closure system clearly doesn't improve that situation. But just as clearly, the change from five-vote closure to three-vote closure isn't aimed at interesting questions that may or may not be on-topic; instead, it is aimed at the flood of uninteresting, obviously off-topic questions that pour into the site each day. Since I have no quarrel with quickly closing questions that fall into that category, I accept the reality that the three-vote closure system is an efficient and popular way to dispose of such questions. Nevertheless, I reiterate my earlier recommendation that the site also consider ways to make it easier to protect interesting, arguably on-topic questions.

One way to do this (which I've mentioned before) might be to grant site participants who have demonstrated a site-defined threshold level of competence and interest in a particular EL&U topic special privileges to prevent closure of questions on that topic that are open but in danger of being closed or to reopen closed questions on that topic single-handedly. Another, less radical option would be to give such site participants the ability to post answers to closed questions, after which they could post a Meta question arguing that the answer provides evidence that the question is on-topic and should be reopened. The request might be voted down, and the question might remain closed, but at least the answer would be in place and visible to future site visitors.

2
  • 4
    I'm with you in your assessment. However, one thing I've noticed , which is definitely an improvement, is that before, when a good question got closed (often because it included a word that had been used in the title of a wildly different question), by the time the question had got reopened it was effectively dead. In other words it had been so long since the question was asked that it had slipped way down the queue and was no longer being visited by users. Under the new regime a good question is often re-opened within a few hours, thus salvaging the possibility of it getting good answers. – Araucaria - Not here any more. Jun 6 at 13:02
  • @Araucaria-Nothereanymore. Based on my experience, this seems to be true. The question english.stackexchange.com/questions/568721/… was reopened in a few minutes once an experienced user helped edit the question to make it on-scope. – hb20007 Jun 8 at 15:05
4

My observation is that low-quality questions are now closed more quickly, while questions that have some substance to them tend not to be significantly affected.

With reopen votes also reduced, it's easier to correct any over-enthusiastic closures. I haven't noticed any close/reopen wars, but consider checking the number of questions that have been toggled between closed and reopened within a short interval (say, a week). If there is programmer bandwidth available, it might be useful to increment the number of close/reopen votes required for a given question each time the question gets reopened.

1
  • 1
    Unlike an edit war, a close war requires 3 new people for each close or reopen. They're pretty rare for that reason. – Laurel Jun 7 at 11:38
-6

I see an awful lot of users not understanding the questions, giving irrelevant answers, and down voting them. When it is explained why the users are incorrect, they double down on their wrongness. It's frankly shocking.

1
  • 3
    I don't see how this is related to the change to how closing questions works. Maybe you should start a new discussion – Laurel Jun 7 at 15:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .