13

As of Sept 22, 11:24am UTC-4, the review queue is at 306:

review queue at 306

This seems outrageous. 209 close items, 37 first posts, a smattering of others (which don't add to near 306)

That's just a lot of work. Going through the Low Quality queue tends to reduce the total one-to-one (nice!).

Going through the Close Votes queue is more like 6 to 1 (20 reviewed, only 3 or 4 removed).

My questions:

  • Is 306 bad? (I think so)
  • How are there so many close suggestions? That is how do they get on the queue? By a single close vote by someone?
  • The others are pretty algorithmic right? First post easy, low quality - some measure of length and other metrics. But does everybody get the same queue? (e.g. if I complete the first post queue, does everybody else's queue go down? (5 votes are needed to close, what about the other queues?)
  • To get them off the close queue, we need more people reviewing. How do we do that?

Update: Wow...thanks everyone for paying attention to this...as of a week later from posting, we're down to ~70. Keep going!!

Update 2 an hour later: Wow...WTH. Did I say something wrong?

Many late answers

(this seems like a crazy jump. Someone must be doing something special.

28
  • 3
    I have to admit I am discouraged by the CVRQ backlog, which in turn dampens my motivation, which in turn (even if insignificantly) exacerbates the backlog... I do wonder if someone is going through super-old questions and flagging or CVing them. It would take quite a while for us to see 300 new questions needing closure. Or maybe it's anon users flagging (inappropriately) new content.
    – Dan Bron
    Sep 22, 2015 at 16:59
  • What @Dan said. Noting the size of the CVRQ lately, I often try to do my bit in clearing it. But it can get very frustrating later in the day, since I've often used up all my closevotes (without seriously denting the CVRQ! :) so I get extra ticked off when I find that I can't closevote new questions as and when they're posted.
    – FumbleFingers
    Sep 22, 2015 at 17:32
  • @FumbleFingers I do sometimes feel guilty for not helping more. One issue is that the mobile app doesn't support reviewing, for whatever reason. Another is that my experience has been a lot of the questions in that queue are aging or aged, and it doesn't seem super important to close questions which no one will ever see anyway. Another is I've had the same experience as you have, and I do like having a reserve of CVs for new terrible questions.
    – Dan Bron
    Sep 22, 2015 at 17:45
  • @Dan: Unless I'm missing a trick, SE doesn't provide any mechanism for me to know how many closevotes I have left at any given moment (until I actually run out). What I'd really like would be a nice clear continuously-updated "Closevotes Remaining" box while I'm working on the CVRQ. Or maybe the ability to configure how many closevotes I'd like to "reserve" for use later in the day, rather than throwing them all at the review queue.
    – FumbleFingers
    Sep 22, 2015 at 17:45
  • 5
    I find it discouraging when I start using the review queue, and come across a question that I don't have any strong opinions on, or worse, that I'm unsure should be closed. Like other people said, I don't want to waste my close votes; I also don't want to vote to close a question that should have stayed open, or prevent a question that should be closed from being closed. I find it frustrating that there's no way to skip questions to come back to later; the "skip" option apparently removes it from my queue. I don't really like using the queue format; I'd prefer to just see a list of questions.
    – herisson
    Sep 22, 2015 at 18:58
  • 2
    @FumbleFingers There is a visual 'close votes remaining' when you are in the review queue and are about to vote to close.
    – Mitch
    Sep 22, 2015 at 22:15
  • I've heard that close votes expire. So if somebody happens to go through old questions and plunks one vote to close on it, if nobody notices or cares, then eventually it will expire and go away and that question with only one vote should come off the queue. So that leads me to believe that, if all this is correct, that someone is constantly using all their close votes every day on old questions that no one cares about.
    – Mitch
    Sep 22, 2015 at 22:17
  • 4
    OK, it seems like I can return to questions I have skipped by looking them up through my review history with the "show skipped reviews" checkbox checked. I just learned about this feature now! meta.stackexchange.com/questions/221134/…
    – herisson
    Sep 22, 2015 at 23:11
  • 1
    The high number of reviews is discouraging. A couple of days ago, I reviewed low-quality questions (I think there were 5 or 6); by the time I was finished, I had reviewed more than twice that number. This is not rare anymore. I frequently just give up before my close votes are depleted, simply because some are iffy and they seem neverending in supply.
    – anongoodnurse
    Sep 23, 2015 at 5:02
  • 2
    The problem, in my opinion, is the overly high quality expectation on this particular SE community, and I believe that this is partly due to the distinction between a community for English and English learners. Is there a distinction between programmers and new programmers? Is there a distinction between TeX and new TeX users? Is there a distinction between travelers and new travelers?
    – JJM Driessen
    Sep 25, 2015 at 15:25
  • 4
    @JJMDriessen I don't think this site has a high quality expectation, in fact it needs higher standards! We're only closing 40% of the questions, but at least 58% of the other questions need to be closed too.
    – curiousdannii
    Sep 26, 2015 at 1:22
  • 1
    lol you think 300 is bad? SO has 8.4 thousand.
    – Lightness Races in Orbit
    Sep 26, 2015 at 11:11
  • 3
    Just a minority view, but this could also be seen as 'too many questions (especially old ones) appearing on the CV list' (& not just as “so many to close, so little time/votes to close them"). Like @DanBron I “wonder if someone is going through super-old questions & flagging/CVing them. Or maybe…anon users flagging(inappropriately) new [& old] content.” I add [& old] re anon flags 'cause I wonder if some high-repers are flagging as anon users to beat the 25/day max. Putting a statute of limitations (1 yr?) on CVing questions & limiting anon flags to comments/answers would address these issues.
    – Papa Poule
    Sep 27, 2015 at 15:54
  • 2
    @PapaPoule Absolutely not. Less people look over old questions, so any non-constructive answers have a higher chance of not being seen. If an old question attracts lots of bad answers it is essential that it be fixed or closed (or protected or something). If there is to be any automatic status for old questions, then old questions should be automatically closed. But the site would be worse off that way, as then duplicates for old questions would have to be allowed. The system as it is now works well, we just need more people reviewing and close voting.
    – curiousdannii
    Sep 29, 2015 at 6:18
  • 2
    Regarding update 2: this is a network-wide problem, the software seems to have created hundreds of spurious 'Late Answers'. See, e.g., meta.rpg.stackexchange.com/q/5819/2451.
    – Tim Lymington
    Sep 29, 2015 at 21:39

5 Answers 5

Reset to default
9

Things get in the CVRQ by having a close vote cast or a close flag cast. They stay there until they get enough Leave Opens or Closes, or are closed, or expire.

It was 322 when I last looked at it. We average two days to close things here:

Over Closed Asked Percent
 1d     12     65 18.46 % 
 2d     51    129 39.52 %
 7d    174    435 40.00 %
14d    363    915 39.67 %
30d    769   2036 37.77 %
90d   2455   6284 39.07 %

One possibility would be to ask the CMs to raise our close-vote count from 24 to 50 the way it is on the bigger sites. We might also need to change our review-queue max from 20 to 40 for that queue. (I have a few reservations about us actually doing that, but not sufficiently well thought out to put into coherent words quite yet.)

Otherwise the close vote queue will just keep growing. At 1,500% of any one person’s quotidian quota of close votes, it is really really hard to keep up.

23
  • raising the size of each individual's arsenal sure is good, but that is a lot more work for a single person to do...and even then it takes 5 people that same amount to finish a close (and more than half are closable).
    – Mitch
    Sep 22, 2015 at 16:49
  • I remember a while back when I'd be surprised at the review queue number even showing up, and that was just for a single edit review.
    – Mitch
    Sep 22, 2015 at 16:49
  • 5
    To put it bluntly, there are too many sh** questions on EL&U, and too many questions (albeit valid they are nearly always poorly formulated) asking about the same things day in and day out.
    – Mari-Lou A
    Sep 23, 2015 at 10:36
  • 2
    @Mari-LouA my thoughts exactly. I was wondering why blatant duplicates never seem to be closed here, but after seeing this question the reason is obvious. What is needed is not fewer review tasks but rather many, many more people willing to review in relation to the size of the site. In comparison, things are perfectly fine at my home site, Mathematica.SE: 7 people have done 20 reviews today and the review queue contains 2 items only. Here, probably about 20 people need to review every day to maintain a steady state; more to fix the current problem.
    – Oleksandr R.
    Sep 23, 2015 at 11:48
  • To make the comparison more explicit: English.SE has twice as many questions (and questions per day) as Mathematica.SE, but nearly five times the number of users. So to keep the review queue manageable should not be difficult if people are doing review tasks regularly. Given that, an important question seems to be: how and why did this situation arise in the first place?
    – Oleksandr R.
    Sep 23, 2015 at 12:28
  • 1
    @OleksandrR.: I question your conclusion. We may have five times the number of users, but that doesn't mean we have that many dedicated users. The problem is only a few of us ever try to close questions, and this can lead to fatigue and a sense of hopelessness. Why bother if my close votes are mostly wasted on dumb questions that will remain open forever?
    – Robusto
    Sep 23, 2015 at 12:44
  • 2
    The way I see it, Oleksandr is very much on the same page with us here. "What is needed is many, many more people willing to review in relation to the size of the site." That's basically the same thing Robusto says.
    – RegDwigнt
    Sep 23, 2015 at 12:47
  • 2
    The one bit I'd object to is the "blatant duplicates never seem to be closed here". Quite the opposite is the case. In fact, duplicate is our top close reason, and has always been.
    – RegDwigнt
    Sep 23, 2015 at 12:48
  • 7
    I actually get discouraged by the review queue (especially the close vote one) for the opposite reason: it's usually full of questions being closed for totally inapplicable reasons, or questions that just need a bit of editing that instead get a sh!tload of hate.
    – Marthaª
    Sep 23, 2015 at 18:34
  • 1
    @Marthaª That makes two of us.
    – deadrat
    Sep 24, 2015 at 10:48
  • 5
    @Marthaª what's stopping you from making those edits?
    – Mari-Lou A
    Sep 24, 2015 at 10:53
  • 4
    @deadrat Hah. If you really agree with Marthaª then why do you only have 39 edits? Why don't you vote more than twice a day?
    – curiousdannii
    Sep 24, 2015 at 10:53
  • 3
    @Mari-LouA: the hate. Well, that and time. But I'd be more likely to find the time if I didn't have to fight the hate.
    – Marthaª
    Sep 24, 2015 at 14:38
  • 2
    @Marthaª If you see hate then flag and report it.
    – curiousdannii
    Sep 24, 2015 at 14:51
  • 2
    @curiousdannii Hah? What do you think you've discovered? That I don't really agree with Marthaª? I'll be happy to explain every action I take or refrain from taking, even though I don't see how any of it reflects on my feelings about the close queue. (Actually, I think a "shitload of hate" is a bit of an overstatement; "carload of dismissiveness" is closer.) I edit to remove distracting formatting and typos. I rarely downvote or edit for substance (except for my own answers). I leave comments instead. Does that help?
    – deadrat
    Sep 25, 2015 at 1:34
8

This is perhaps an inevitable affliction of mature Stack Exchange sites. The closure review queue grows due to a confluence of several factors.

I have found that empirically, user reputation scores follow a power law distribution, or something close to it. Here is a histogram; the same histogram plotted on a log-log scale looks roughly linear. This means that:

  • Counterintuitively, the median reputation of the entire population decreases, even though the reputation score of each individual member tends to increase.
  • The growth in the number of new users exceeds the growth of the number of users who have earned 3000 points.

As the database grows, the probability of asking a good question that isn't a duplicate decreases. (The site still gets a reasonable number of good questions, since the increase in volume can mask the problem.)

For a brief while after earning the vote-to-close privilege, it feels nice to wield the power to close off-topic questions. However, the novelty soon wears off, especially when the queue has grown so large that reviewing is more a demoralizing chore than a privilege.

What can be done? Some ideas:

  • Lower the eligibility requirement below 3000 points to keep up with the falling population distribution of scores.
  • Lower the threshold for closure to some number below 5 questions. It would be interesting to find out what value is added by requiring a 4th or 5th vote — how many questions get 4 close votes but never get the 5th vote? I don't believe that it's possible to extract that information from Stack Exchange Data Explorer, so we would need to ask staff for the statistics.
4
  • It seems unlikely that either of those two things will ever come to pass, given that they are not measures that have ever been taken before.
    – tchrist Mod
    Sep 24, 2015 at 13:42
  • 2
    This may be an inherent weakness in SE's reliance u:pon search rather than curated/classified FAQ documents. A random language question, even when substantially similar to a past question, may be a sufficiently different set of words that a simple search has no chance of finding the answer. In the technical areas, limited vocabulary may help resolve that, but here it turns into a natural language parsing problem and becomes intractable.
    – keshlam
    Sep 25, 2015 at 3:10
  • I think Software Engineering is currently experimenting with your latest suggestion. The results of that experiment may or may not influence this topic.
    – Mast
    Nov 30, 2015 at 23:20
  • 2
    @Mast The experiment on Programmers
    – 200_success
    Nov 30, 2015 at 23:29
5

This site gets lots of new users who post questions which aren't up to scratch. I would guess that a long term average of 40% closed is on the high end across the network.

Perhaps something that could be considered would be a site-specific lowering of the number of votes required to close questions. If questions needed only 3-4 close votes the queue would be processed considerably quicker.

5
  • 2
    We are at #6 network-wide in the close-vote percentage leagues. The median is in the low 20s.
    – tchrist Mod
    Sep 26, 2015 at 1:28
  • 1
    I think it's more important to get closures right than to process them quickly. Lowering the vote threshold just to rush through the queue faster seems like a dangerous proposition and a slippery slope. Kind of like saying we can reduce the workload and waiting period during a general election if we start letting only those living in the capital city actually vote.
    – Lightness Races in Orbit
    Oct 2, 2015 at 8:19
  • @LightnessRacesinOrbit However because close votes expire, if they stay in the queue for too long then that isn't doing them right. A possibility (though the complexity would probably mean it's not worth it) would be for the close vote quota to be reduced only if no 'keep open' reviews have been made.
    – curiousdannii
    Oct 2, 2015 at 8:22
  • @curiousdannii: Doesn't it take like a month for them to expire?
    – Lightness Races in Orbit
    Oct 2, 2015 at 8:26
  • @LightnessRacesinOrbit I thought only one or two weeks. Probably they only expire on disputed questions, but on slow sites it happens occasionally even on obviously off-topic questions. (There is a much longer period before you're allowed to vote again on the same question.)
    – curiousdannii
    Oct 2, 2015 at 8:28
4

...as of a week later from posting, we're down to ~70. Keep going!!

Wow...WTH. Did I say something wrong?

831 Late Answers

StackExchange implemented a change to the way they evaluate late answers:

Can we raise the bar for reputation for late answers to bypass the review queue?

TL;DR: As of today (September 29, 2015) the maximum reputation for having an author's late answer enqueued has been raised from 10 to 50.

That would be roughly when you noticed the jump. I think it is affecting all sites - suddenly you get heaps of entries that need "reviewing" - some months old.

2
1

Gold Tag Badge holders and Mods can insta-close things. If this is still a problem, I'd recommend figuring out if there's any specific tags that have a disproportionate volume of close votes and specifically making a call-out to gold-tag-holders there to help with question closure. Possibly you have a category of questions you are currently allowing that are too much work for the stack to maintain and they should really be made off-topic entirely.

Alternatively, maybe you just need to add an extra mod to the mod team, given how big your site has grown.

3
  • This site has only awarded 30 bronze tag badges, 3 silver tag badges, and no gold tag badges, as of September 2015. (meta.english.stackexchange.com/help/badges)
    – Jasper
    Sep 30, 2015 at 17:46
  • @Jasper Well! Perhaps we should find out what tags need gold tag badgers, and then make a point of paying attention to (and appropriately upvoting Q&A in) those tags, even if we normally don't pay much attention in those areas.
    – Please stop being evil
    Sep 30, 2015 at 18:04
  • 3
    I think the gold tag badge holders can only insta-close as duplicate.
    – curiousdannii
    Oct 1, 2015 at 3:38

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .