16

Recently I posted an update which asked

UPDATE Sunday 23 June, 2019 […] Is it just me or has the quality of questions by new contributors dropped even further? Is there a way to measure this statistic?

Furthermore, speaking purely for myself, I have greatly reduced the number of contributions I make to the site because of LQQs and …blah, blah, blah …

Well, I have found the query that tells me the answer. It ain't good news.

   English Language & Usage

enter image description here

Compare those results with

   Worldbuilding

enter image description here

and with

   Puzzling

enter image description here

What about Stack Exchange's flagship, how is that faring? Oh, much worse, much worse but it is by no means a consolation

   Stack Overflow

enter image description here

Thoughts and prayers? Yikes.

  1. Can anything be done to stop the erosion of quality on EL&U? Is it too late? Can we come up with a good idea that will foster better and/or more interesting questions?
  2. Why is the community upvoting fewer questions? Why are fewer hi-rep users posting questions? (Don't all look at me)
  3. Does the blame lie solely on Low Quality Questions (LQQs) or have we become a jaded and cynical band?

As for EL&U's ‘oldest’ post, it was migrated from Stack Overflow on March 13, 2012 but first asked Jun 16, 2009 and it just turned 10 years old. Sugar, we should have thrown a party.

36
  • 2
    It’s interesting that the trend is always down, probably due to older questions having more opportunities to get voted on, although how fast it drops varies. Maybe people inclined to ask good quality questions find their question has already been asked and answered, so they up-vote the old question instead of asking a new one. Folks inclined to ask lower quality questions don’t bother looking.
    – ColleenV
    Jun 24, 2019 at 21:56
  • @ColleenV puzzling and worldbuilding have kept consistent average scores.
    – Mari-Lou A
    Jun 24, 2019 at 22:03
  • 5
    @ColleenV the truth of the matter is we've been complaining about LQQs on EL&U for a number of years and it just keeps getting worse. Nothing has been done to make the site enjoyable. If our best users are leaving, that also means the number of quality answers shrinking. I'm not learning nearly half as much compared to four years ago. I've reached a plateau.
    – Mari-Lou A
    Jun 24, 2019 at 22:08
  • 2
    Can you run your query for all EL&U questions tagged 'single-word-request' and for all EL&U questions not tagged 'single-word-request'? If you did, I'll bet you'd find the results even more depressing.
    – Sven Yargs
    Jun 24, 2019 at 22:30
  • @SvenYargs Oh, I didn't create the query, I found it ready made, maybe someone here can tweak it. P.S. I fixed the broken link now.
    – Mari-Lou A
    Jun 24, 2019 at 22:32
  • @SvenYargs Query split by single-word-requests -- not sure how conclusive those results are.
    – Andrew Leach Mod
    Jun 24, 2019 at 23:12
  • @AndrewLeach: Thanks much for the queries. The results aren't as far out of balance as I would have predicted, which is actually quite heartening. I'm also curious about the corresponding upvote numbers for answers to SWR-tagged questions and answers to not-SWR-tagged questions. Would running those queries involve a simple change in the query wording that you already have, or would it be a completely different undertaking?
    – Sven Yargs
    Jun 24, 2019 at 23:24
  • @AndrewLeach my conclusion is that SWR are disproportionately more likely than other tags to get bumped to HNQ. As a result, they end up with higher question scores. The reason for getting them on HNQ is that they tend to be broad and easy to answer meaning they tend to get multiple answers shortly after being posted thus outperforming harder questions which require more time and effort to answer.
    – JJJ
    Jun 24, 2019 at 23:38
  • 1
    Does this query exclude closed and duplicate questions? Is that what the post type ID does? Frankly, the only way to prevent these sorts of drive-by “solve my problem but I don’t want to be part of the community” is to prevent new users from asking questions. Make them build up reputation with answers first. That’s throwing the baby out with the bath water though.
    – ColleenV
    Jun 25, 2019 at 12:08
  • 2
    I am skeptical as to the meaning of low voting numbers. I don't think it is necessarily directly linked to poor quality content that people react to with well thought out votes. It could very well be simply a lack of interest in voting at all as the site ages and the community slowly evolves changes (older users dropping out, newer ones coming in). I'm not sure right yet how to quantify that difference with data though.
    – Mitch
    Jun 25, 2019 at 13:32
  • 2
    @Mitch I think at a minimum the average score should be weighted by views, with highly viewed questions counting for less. It might be interesting to analyze questions that were actually flagged low quality to see how many of them there are, how they came to be, and how they were disposed of.
    – ColleenV
    Jun 25, 2019 at 13:37
  • 1
    Here’s an interesting graph of site activity (questions asked, votes on answers and questions, et. al.) data.stackexchange.com/english/query/161411/… Questions seem under-appreciated compared to answers.
    – ColleenV
    Jun 25, 2019 at 14:54
  • 1
    Can you give comparable stats for ELL? (it seems to be a point of comparison in the comments)
    – Mitch
    Jun 25, 2019 at 16:09
  • 1
    Mari-Lou, @ColleenV: ELL is doing 'worse' in the comparable statistic. And I think your 'Yikes!' really should be a concern for Stack Overflow. That site is a world resource that the world's programmers know about and use daily. (ELU is not a resource for anybody in the news; for language questions, Quora would be checked first by a journalist.) So if the world is ending for ELU, we'll see others' demise first. (this comment doesn't answer any questions, just adds more data.)
    – Mitch
    Jun 25, 2019 at 19:07
  • 3
    The general low levels of upvotes on ELU has been lamented before. Users, probably jaded and cynical, appear to direct their efforts more versus CVs and DVs for reasons that are not always clear. Efforts to save questions that can be easily fixed to be on topic are generally seen as suspect and unwelcome. LQQs flow could be easily controlled implementing some sort of filters, but that still is paradoxically is seen an unwelcoming while downvoting and “invite” new users to read the rules section is not!! Do we all need therapy?
    – user 66974
    Jun 26, 2019 at 11:03

4 Answers 4

3
  1. Can anything be done to stop the erosion of quality on EL&U? Is it too late? Can we come up with a good idea that will foster better and/or more interesting questions?

Easy solution: confirm my HNQ hypothesis. Then consider the option to ban from HNQ. In the new HNQ system, the mods have the authority to remove questions from HNQ. A community consensus could be reached to authorise mods to remove any question from the HNQ. This is a good option because it will push other interesting ELU questions to HNQ.

  1. Why is the community upvoting fewer questions?

This is not an indication of fewer upvotes in absolute terms. For that, we would have to look at absolute vote numbers. As an extension, we could look at vote numbers by active users (for each year consider only users active in that year) and plot over time.

  1. Does the blame lie solely on LQQs or have we become a jaded and cynical band?

Without having looked carefully at the data to support this, I think this is a sensible hypothesis. Furthermore, there's a domino effect. If quality goes down, other things go down too (fewer new users with subject experience stick around, settled users leave or are less satisfied than they would otherwise be).

10
  • 2
    Removing SWRs from HNQ will mean lowering the average score, surely? If today the average score per question is 1.2 upvotes that number will probably be lower if SWRs are "banned". It is SWR that gets eyes on the site because 1. they're (sometimes) fun and/or intriguing to read and 2. it's relatively quick and easy to see which answer is the best fit. I don't think SWRs are to blame, I think the number of questions that are about meaning are the worst culprits…
    – Mari-Lou A
    Jun 25, 2019 at 4:00
  • 1
    …followed by grammaticality. Note on the first page of grammaticality a list of zero upvotes and posts with -1 and -2. Yet both type of questions are on-topic and should, in theory, interest serious English enthusiasts (experts).
    – Mari-Lou A
    Jun 25, 2019 at 4:02
  • Users are not voting on questions tagged grammaticality, grammar and meaning. Just look at the volume of unregistered users and new contributors stuck at 1 rep. That is not healthy. Banning SWRs or merging with ELL (which I fully disagree with) will not get rid of the problem. What is happening to EL&U?!
    – Mari-Lou A
    Jun 25, 2019 at 4:11
  • @Mari-LouA agreed with statements on those other tags. The thing about removing SWR from HNQ is that those spots are now open to other questions. That said, your first comment seems circular and it may mean bumping slightly slower questions that are more interesting. And let's face it, SWRs are, especially once bumped to HNQ lists of suggestions. That's not bad per se, some sites are based on that, but English could offer more (I think).
    – JJJ
    Jun 25, 2019 at 9:25
  • There is an algorithm for HNQ, and removing a question which meets its criteria does not mean that a different type of question will get in. Even if all SWR questions are automatically excluded, there's no guarantee that anything else will meet the inclusion criteria. (I've removed one, maybe two, clickbait questions from HNQ, but really we don't get a question there very often these days.)
    – Andrew Leach Mod
    Jun 25, 2019 at 12:10
  • @AndrewLeach I don't think that's true. Unless the new algorithm works vastly differently, it still allocates (in practice) two questions per site. Currently, that's 502821 and 502706. The algorithm assigns a score to each question within our site and the top scoring question gets no penalty, and subsequent ones get an increasing penalty (so it's not one site that dominates HNQ). I assume that removing our top question will remove the penalty on the second and lower the penalty on the others so the previously lower scoring questions slide in.
    – JJJ
    Jun 25, 2019 at 12:53
  • @ColleenV like I said, it was more of a comment taking into account the bigger (SE-wide) picture. In the end, I don't think either site is working toward building a complete repository (compared to e.g. Math.se). As you may or may not know, I don't really participate on main here anymore because it's just a rep-farm around SWRs. I'm all for SWR, but there should be some balance between the tags.
    – JJJ
    Jun 25, 2019 at 13:00
  • @ColleenV in my comment (it's just an idea, no need to get into a fight over for either of us ;)), I suggested a bit more. Basically merge ELU and ELL (into one general purpose English site, for everyone). Then make a new site which is aimed at more academic questions where one can discuss more in-depth at publication level (like mathoverflow). Maybe you're right though, maybe there's no appetite for such an academic site and it would just drag ELL down with ELU.
    – JJJ
    Jun 25, 2019 at 13:20
  • @ColleenV I'll remove that part from my answer, it distracts too much and doesn't really help the OP I guess. The rest of this answer still seems sensible to look into.
    – JJJ
    Jun 25, 2019 at 13:22
  • @ColleenV fair enough. In that case it would still be an option to split ELU into two so there are three English related sites. Not sure if that's something people will be happy about so I will just leave it as comment. ;)
    – JJJ
    Jun 25, 2019 at 13:24
3

We need to accept the fact that the only way to significantly reduce the flood of low quality questions is to not allow users to post until after they’ve earned some reputation. That, however, has unwanted side-effects and is not within the community’s power to implement.

I don’t think getting rid of the low quality questions is as important as attracting and elevating the sorts of questions that are desirable. Some sites do scheduled events to generate and promote content or encourage a particular activity. Maybe organizing an event for the community will work better than Mari-LouA trying to single-handedly to seed the site with quality questions.

I don’t know exactly what that event should look like- whether it should it be asking questions, or finding gems in the rough, or swarming the low quality questions to get them closed- the important part is to get the community rowing in the same direction and making use of the power we do have to affect the site. If the community doesn’t respond, then we have a better idea where the problem is, and can start working on some community building solutions.

Mi Yoda has a “best answer” contest, why not a “best question” contest? It probably shouldn’t be score based, but maybe nominated and voted on in meta, to avoid the HNQ effect?

There is no silver bullet that will solve this problem in one fell swoop. The underpinning of a healthy SE site is the community that curates the content. If it’s too small and the amount of content coming in is too much for it to handle well, that’s a problem. If it’s large enough to handle the content, but everyone has different ideas about what kind of site it should be, that’s a problem. If there are a lot of members, but they aren’t engaged with the site and actively curating content, that’s a problem. All of those problems may have similar looking symptoms and completely different root causes and solutions.

I don’t know the solution, but neither does anyone else. We have some guesses, but until we actually try something and see what impact it has, we won’t know. Trying something will give us more information about the problem than speculation ever will. It doesn’t matter if it’s this specific idea, but we need to organize a committed effort to attempt some incremental step toward solving some aspect of the problem instead of trying to come up with the perfect solution in theory.* It will be easier for us to try things if we start with things that we can do as a community without much assistance from the SE team.

* Yes, the programmers here will think “Hey, that sounds like Agile”. I am in fact a certified scrum master. I am willing to either Kanban or Scrum the shit out of almost anything

11
  • I think events are a fun way to build community and create good examples, but it’s not sustainable to create a meaningful and long lasting repository of answers and a large presence to ensure a reliable flow. My proposed solution is twofold: one, get the site in a shape attentive to the kind of audience we strive for (this includes stopping and questions before they start, as you say, as maybe events as you say, at least to have a decent “face” when people show up), and then directly and consistently market to this audience, eg LanguageLog, by improving SERPs through exclusion of LQQ, etc.
    – Dan Bron
    Jun 25, 2019 at 14:34
  • @DanBron I agree that there is probably some community building and marketing that needs doing as well. Events can help strengthen the existing community and maybe encourage a few passersby (passerbys? Hrm) to be more engaged. If an event can also partially address some of the content issues, that would seem like a good place to focus some energy.
    – ColleenV
    Jun 25, 2019 at 14:41
  • 2
    @JJJ It is not general purpose, the audience was mooted and clearly identified: linguists, etymologists, and serious English language enthusiasts. IE word-nerds, people invested in studying English qua a language. The idea that it’s “general purpose” is precisely the myth that we have to quash, and also convince Google’s SERPs to quash thorough diligent curation and prevention and removal of “general purpose” Qs, and encouragement of interesting, deep questions. Like the LanguageLog and similar blogs for lovers of English as a language.
    – Dan Bron
    Jun 25, 2019 at 15:30
  • 1
    @JJJ The lack of focus on what ELU was originally intended to be has been a problem for a while. ELL was split off because helping language learners is definitely not what ELU was intended to be. SE doesn’t need a “general purpose” English site. It will be filled with homework and test prep. Maybe SE needs a single word request site, since those questions are popular, but don’t exactly need expert English knowledge to answer.
    – ColleenV
    Jun 25, 2019 at 15:45
  • 1
    @JJJ It’s not just “enthusiasts”, it’s “serious English language enthusiasts”, with an emphasis on serious and a focus on* language*. This population, in the company of the more academically trained linguists and etymologies, paints a very clear picture of the intended audience for the site. One needn’t be academically trained to be a serious enthusiast, one can be an armchair linguist, read Safire or language log, pop works on the subject. An avocation as opposed to a vocation. But the focus f such people will be different fr people wanting homework help, writing a resume, article usage..
    – Dan Bron
    Jun 25, 2019 at 17:29
  • @JJJ Re: your initial comment- on ELL we have seen some improvement when we’ve taken the time to show people how to meet the quality standards by example. We see newer members picking up those ideas, even when they weren’t the people we helped. The people who are interested in joining the community will look around to see how best to fit in, so a collection of exemplary or prize-winning questions could be a good starting place for them. If we look at ELU sorted by score, is that the image we want? Or would a curated collection of questions that reflect the intended spirit of ELU be better?
    – ColleenV
    Jun 25, 2019 at 18:25
  • 1
    @JJJ Haven’t we already been doing that? At some point you have to have content that attracts the serious language enthusiasts and keeps them engaged. On ELL, our community is looking for help with their English, so we help new users write their questions and show them how the site works. They got what they were looking for: help, and it often makes them interested in being part of the community. What are the desired community members of ELU looking for? We should give it to them when they show up.
    – ColleenV
    Jun 25, 2019 at 18:52
  • 1
    @JJJ I enjoy the etymology questions as well, and it’s possible someone experienced in etymology might enjoy answering them, if they’re well-asked. So maybe we need more well-asked etymology questions to hit the HNQ or show up in search results. Maybe the event is a contest for the most interesting etymology question asked this month and we keep track of the winners on meta so we have a list of interesting questions we can use to get people we bump into who are interested in etymology interested in the site.
    – ColleenV
    Jun 25, 2019 at 19:12
  • 3
    Why can't we showcase some of the best answers on EL&U somewhere? Not just the most upvoted. Select a top ten of the best etymology posts. Top ten of SWRs. The best head scratching grammar questions? The most surprising facts? The weirdest spelling histories. Why can't we make knowledge not only more accessible but also "fun"?
    – Mari-Lou A
    Jun 26, 2019 at 2:13
  • I sometimes think there's enough material to make for a really informative, eye-opening, mind boggling compendium. But it's all hidden in the archives. We used to have easy access to "votes" but that page is no longer prominent, you have to click half a dozen times before finding it. On a mobile, for some reason, the menu item "votes" is almost meaningless. We lost viewers and users when the responsive design was adopted. There, I said it! On metaSE I was given a warning for mentioning it in a "tangential" comment. I trust the same won't happen here.
    – Mari-Lou A
    Jun 26, 2019 at 2:16
  • @Mari-LouA We can make such a compendium, the first step is marshaling a posse to identify the questions. As you say they’re buried and there is no possible automated way to find them. We can start simple: ask a Meta Q on this topic, inviting answers to nominate particular Q/A as interesting and describe why the person thinks their interesting. Let the Meta community vote to decide which should be showcased. We then take it from there: decide how to display the collection, maintain it, etc. but it starts by doing the dirty and hard work of finding the Qs first (I’m not volunteering ;).
    – Dan Bron
    Jun 26, 2019 at 12:24
2

I'm not contesting your observation that question quality is dropping, but I'd like to offer a different view of the data. As @ColleenV notes,

It’s interesting that the trend is always down, probably due to older questions having more opportunities to get voted on, although how fast it drops varies.

so I thought it would be worthwhile to check how 2019's questions are doing compared to, let's say, the situation for 2018's questions on June 23rd, 2018, and so on. That's what this SEDE query does, and the trend is indeed downwards. Also, Roomba hasn't yet cleaned up abandoned questions from 2019, which might influence the average (SEDE doesn't take deleted posts into account).

enter image description here

As for the other sites in your question, Worldbuilding shows an upward trend, Puzzling's progress seems to have stalled and Stack Overflow is (very) slowly going down.

8
  • Yep. found them english.stackexchange.com/site-analytics
    – Mari-Lou A
    Jun 26, 2019 at 13:43
  • Is there a way to visualise how a question upvotes changed over the course of 7, 6, 5, 4 and 3 years? Was the bulk of the upvotes cast in the same year the question was posted or did those upvotes accumulate over the years? For example, many of kiamalaluna's questions are highly upvoted. (english.stackexchange.com/users/252/…) Was that true the very first year or is that a result of votes cast over the years?
    – Mari-Lou A
    Jun 26, 2019 at 14:38
  • I'll have to think about the visualization, that is not really my forte.
    – Glorfindel
    Jun 26, 2019 at 14:59
  • In case day of the week matters (as I think it might), I note the following day of the weeks for each of the years tracked in Glorfindel's chart: Thursday 2011, Saturday 2012, Sunday 2013, Monday 2014, Tuesday 2015, Thursday 2016, Friday 2017, Saturday 2018, Sunday 2019.
    – Sven Yargs
    Jun 26, 2019 at 16:08
  • @Mari-LouA something like this? It list for each year of the question (those are the different lines) how many upvotes it received while being 0, 1, 2, etc. years old. The last data point of each line is misleading, since not all questions from 2010 are 9 years old.
    – Glorfindel
    Jun 27, 2019 at 11:12
  • The lines are a little cramped, and the graphic's a bit confusing . How come 2010 is so high above the rest?
    – Mari-Lou A
    Jun 27, 2019 at 12:41
  • The 'best' questions always get asked first, I guess ... Even on sites where we see the question quality (as measured by # of upvotes) is increasing, like Worldbuilding and Puzzling.
    – Glorfindel
    Jun 27, 2019 at 12:44
  • The 'best' or the most common and basic questions? I don't know, the graph is difficult to decipher. Maybe take one question between 2010 and 2012 and plot its progress through the years?
    – Mari-Lou A
    Jun 27, 2019 at 15:29
1

Can anything be done to stop the erosion of quality on EL&U? Is it too late? Can we come up with a good idea that will foster better and/or more interesting questions?

Why is the community upvoting fewer questions? Why are fewer hi-rep users posting questions? (Don't all look at me)

Does the blame lie solely on Low Quality Questions (LQQs) or have we become a jaded and cynical band?

I have a few suggestions.

1) I like Mari Lou's idea of showcasing answers, and suggest that you might want to go further by taking a stab at fulfilling the goal of providing a source of definitive answers on the English language and its use. Right now searching is limited mostly to strings, making it hard to find answers suitable to questions.

Thus, individuals with expertise in a particular area might want to curate the questions and answers in a given area and provide a summary article with links to the relevant q&a's. This amounts to writing an annotated Table of Contents for a book and filling in the content with links to existing material.

How to do it? Maybe like this:

--The volunteer for a particular chapter creates a question (or the question is asked by an account called "English Enthusiast") and answers it himself or herself. The question and answer define the topic, cover misconceptions, and break the topic down into subparts with introductory/summary material and curated links to good questions and answers.

For example, supposed John Lawler could be persuaded to answers sometime like "What's with these modal verbs, anyway?" This answer then defines modal verbs, covers misconceptions, and goes on to explain. (What is a modal verb and what's happening to them?) (Ten things about modal verbs your ESL teacher got wrong)

Etymology (and the major fallacy: what you don't know about etymology that can hurt you) is another topic someone could take on that's not so broad that it becomes impossible to handle. Inevitably the "answer" is going to include a list of links to other questions.

Another Q: I'm determined to use a, an, and the properly--but I also want to know when I can go naked.

(I think a little humor or jazz is important; the audience is, after all, probably under 25.)

Developing a list of people willing to take on a slice of what's already on EL&U, and discussing how to circumscribe the "chapters," is the first task. But the whole thing need not be done at once.

The virtue of the above is that it follows the standard SE format of q&a. A link to an overview document on the questions asked by "English Enthusiast" could be placed on meta. It would also reveal areas of weakness where new questions needed to be asked. Second answers would be available for additions and updates.

2) The LQQs generate a lot of work. Possibilities to reduce them:

a) Refrain from answering the questions in a comment, even to help out and then vote to close. It is so tempting. I am guilty, guilty, guilty. If I don't do it, someone else will. And one wants to help. Yet, it gives the drop-ins exactly what they want: a quick (and usually quite expert) answer. I know tchrist campaigned on this for a while and gave up. But no one really disagrees about the consequences of feeding stray cats.

b) Automate the closing of questions by closing anything that's been open and unmodified for 48 hours and has nothing but "off topic" and comments--or something to that effect. Get from the in basket to the out basket not necessarily faster but with less work.

EDIT

3) Consider, while resolutely ignoring questions that don't belong on the site, opening up a little in terms of what's on-topic. Some questions come up about how to become more fluent in English, what to read that's fiction without too much dialogue, good references on, e.g., grammar, the meaning of vocabulary or phrasing in current political debates, and the like. Some of these posters are potential contributors, and current contributors have a lot to offer them on these topics. This can be managed by upvoting a question and answering it even though it's a bit of a stretch. EL&U, assuming it survives, will look vastly different in five years than it does now; how do we let it morph?

Oh well.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .