What do we consider "consensus" for burnination?

Question

Bit of a meta meta question.

We've been having a conflab about burninating the correction.

My position was that fourteen votes (at the time the conversation started) wasn't enough to know that community wants the tag gone.

However, this is just a gut feeling. I don't have a solid number of votes and ratio of up to down votes that would shout "burninate it now".

So my questions are:

• Do we need community consensus to burninate a tag?

If we do need it then:

• How many people need to have voted (on the question, or the answers) to say that we should get rid of the tag?
• What should the ratio of up to down votes be to give the go ahead?

I am asking for numbers, but I expect them to be fuzzy thresholds.

Equally, if you can think of other ways to come to consensus, different to the above stats, please propose them, too.

Answer 1

Do we need community consensus to burninate?

I think we do. Why? Because burninating a tag is not just editing it out.

Technically it might be a sum of edits removing the tag from the affected questions. For this we don't need any consensus whatsoever. Every ELU user with 2k reputation has the privilege to edit stuff unsupervised. While trivial edits are discouraged for obvious reasons any 2k member is trusted to wield his or her edit powers responsibly. If a user feels a tag is unnecessary he or she is free to go on a edit spree or edit a few questions at a time. That's an earned privilege. It is also a very good way to stop bad new tags in their tracks.

However that's not what burninating a tag is about. Burninating a tag is about getting rid of a tag that has already a foothold in the taxonomy and doing so with meta involvement. That is why the linked meta post describes the criteria for burninating a tag. It also calls for meta agreement. Such an agreement and several burninations of tags are also a requirement for possibly blacklisting a tag later.

Edit: I split this answer in two, to separate the answer for the question about having a consensus (this answer) from the one what constitutes a consensus (over here).

Answer 2

Disclaimer

This discussion has been primarily started by my assertion about this question posted by me where I assumed +10/-4 to mean a pretty strong sign of community support. Please aim any comments to the specific tag at that question. This post is just about participation on meta and what can reasonably be considered a consensus. Also a warning, lots of numbers ahead.

So let's leave the particulars of the case behind and have a look at the kind of participation we can expect form ELU.meta.

What makes a community consensus?

Observing and participating in meta

For meta to agree on something people have to be given time to read and evaluate an issue, form an opinion and voice said opinion. Thus a certain timespan has to be given to the topic. The timespan has to be long enough to give a reasonable amount of people the possibility to voice their opinion. We have ~480 people visiting meta in a month if I take this month as representative. Not even 300 in the last two weeks. Maybe the 10k users have access to better numbers. Anyhow most questions on meta do not get even 250 views in three months. The average view count of all 2016 questions is 153.

Most internet sites follows the 90-9-1 rule, meaning 90% just lurk, 10% contribute (vote) while 1% answers or asks. I don't know about the actual SE numbers, but this query strongly suggests that this seems not too far off for ELU.meta. There are only a dozen questions all year that outpace those 10% score.¹

With an average view count of 150 and about 10% of people actually voting, 15 votes is what is to be expected on a topic of average interest. Thus, the 14 votes on ~115 views the correction tag question² had before this meta post and the discussion is totally within the range of what is to be expected as participation. It is even a bit above par.

Summarizing, on any common topic about 150 views and around 15 votes are about as much participation as we are gonna get on ELU.meta.

Since this assertion has been challenged by gut feelings in the comments I offer a more detailed analysis.

I adapted a great SEDE query about weekly stats to monthly stats and ran it for meta.ELU. Raw data is here. As we can see there my assumptions of meta participation based on the view count were even too high. Excluding election months, the active meta participants range roughly between 60-90. That's active meta participants, everyone who asked, answered, commented or voted in said month.

I added two divisions in Excel (right-most columns) that show how many votes we can expect per answer or question in a month. Spoiler alert, it's not that many. We really cannot expect a lot more than 10-12 votes on any given question.

I made a second analysis based on this query and added an column for accumulated percentage of questions per vote count. It includes questions since 2015. (Older questions have less votes on average and would as such strengthen a case for low votes being a base for consensus. Thus, I excluded everything older than 2015.)

We can see that almost fifty percent of questions never reach six votes. Two thirds never reach double digit vote counts. With 13 votes a question enters the top 20% and only 10% make it past 17 votes. Note that these are all vote totals, not score totals.

Summarizing, we really cannot expect for a common topic like tag gardening to reach a high amount of votes in total. Not even in the long run. Everything above 12 total votes is in the top 25% percentile and very well voted on.

How to interprete those votes?

If it's one sided the question is irrelevant but in a case were the votes are split we have to establish an agreement about which net score or voting disparity constitutes a favorable vote.

10/4 or ~70/30 in percentages is in most environments considered a strong vote of confidence. Nearly 75% if one counts the OP in favor of the topic which seems a valid assumption on a burnination request. Since meta is our essentially ELU parliament one might excuse the comparison, but in most countries more than two thirds or even three quarters are enough to change the constitution. That should be enough to burninate a tag. Of course percentages are susceptible to low total numbers. However, as I detailed above, we are unlikely to get higher participation on the average topic.

Why +5 is a solid value when disputed

As detailed above we cannot assume to get 17 votes on an average topic. Counting the OP in favor of the suggested burnination +5 means at least two thirds support of interested users.

What's custom?

A last thing to consider is that people tend to involve themselves less in stuff that works. Specifically what was considered a good standard to burninate before? Consider this list of burninated tags³. We have burninated tags (unanimously) with as low as 3 votes and rarely have a score reaching double-digits.

Conclusion

Considering the expected vote count anything that reaches high single digit scores or a net score of +5 when disputed should be considered a vote of confidence unless there is an answer stating something else.

If there's an answer it has to be decided on a case by case basis.

---

_{^{1 I used the score rather than the actual vote count which is obviously higher for disputed topics.}}

_{^{2 Disclosure: I created that post.}}

_{^{3 Disclosure: I created that post as well.}}

Answer 3

I don't think any kind of consensus is needed, except for the consensus that an individual edit to a post is correct. If removing a tag from a post is a correct action, then it requires no consensus to act. If removing the tag from all its posts is correct, then it can be done without a consensus.

In cases where we disagree about whether an edit is correct, or how a tag should be used, then I'd say we need at least a dozen votes. However, I'd also argue that we need a discussion. People need to provide arguments about what is correct or not. Downvotes are not an argument, unless they are overwhelmingly outnumbering upvotes.

People downvote tag burnination for a variety of reasons. One reason is that they don't like to see edits bumping old posts. In my opinion, that is not sufficient reason to avoid editing old posts. Tag maintenance should not be put off just because the "active" tab gets cluttered while the maintenance is going on (although if the StackExchange software were ever changed to not bump posts like that, even this excuse would go away). If someone doesn't provide a reason that a tag edit is incorrect, then their downvote doesn't really count for much.