TL;DR: Shall we support systematic A/B (and similar) testing of QI (Quality Improvement) proposals, or would advocating for such research simply lead to frustration, i.e., would it be an exercise in futility?
I see what you mean...
As an enthusiastic newcomer to EL&U, I quickly noticed a problem most of you have endured for many moons: the plethora of low-quality, help-me-with-this-crossword-clue, single-word questions. I have endeavored to read previous EL&U Meta discussions on this topic, but if my query has been asked and answered before, point me in that direction please. :p)
If not, my question is:
Would you support systematic testing of proposed quality improvement methods?
I acknowledge that there are some 'unknowns' here. For example, I don't know exactly what 'systematic testing' would look like. Developing usability and effectiveness tests would require collaboration with the good folks at User-Experience.SEand Cross-Validated.SE, in addition to the Stack Exchange staff.
My hypothesis is that some potentially great ideas have not been implemented due to uncertainty or disagreement about their utility and effectiveness.
So instead of getting stuck in the deep mud of uncertainty, or in the quagmire of endless debate, let's start systematically testing quality improvement (QI) proposals.
Example #1
In response to a post from @Shog9 on 15 Nov 2011, Single word requests, crosswords, and the fight against mediocrity, @Unreason suggested:
I think the first thing that needs to be done is to make sure that we inform the new user what is expected from them, namely that:
- they should show research or attempt at research in dictionaries and online encyclopedias
- if the research failed that they should explain why or how
There are many ways this can be incorporated into the site (personally, I am particularly fond of the idea that users under X points should click on 'I have done research' with every question they ask).
We can do A/B testing on that idea! ("users under X points should click on 'I have done research' with every question they ask")
Example #2
On 30 Jun 14, @tchrist posted A simple proposal towards improving question quality, wherein he suggested adding a prominent banner to be shown to newcomers on the 'Ask a Question' page:
Have you thoroughly searched for an answer before asking your question? Sharing your research helps everyone. Tell us what you found and why it didn’t meet your needs. This demonstrates that you’ve taken the time to try to help yourself, it saves us from reiterating obvious answers, and above all, it helps you get a more specific and relevant answer!
@phenry offered a banner with different wording and hyperlinks:
Welcome to English Language and Usage! The key to getting a good answer is knowing how to ask a good question. Please see the Help Center for our guide to getting the information you're looking for.
Stack Exchange works best as a supplement to basic research, rather than as a replacement for it. Many basic questions can be easily and quickly answered by consulting commonly available online references. If you've researched your question and haven't found an answer, tell us what you found and why it didn’t meet your needs. Sharing your research helps everyone, and will help you get a more specific and relevant answer!
If you're not a fluent speaker of English, you may get a better answer if you ask your question at English Language Learners, our sister community for speakers of other languages learning English.
There's another A/B test! - Comparing the wording of one banner versus another.
I could post other examples, but I'm running out of steam and I think you get the point.
To reiterate: Would you support systematically testing various strategies to improve question quality?
Thanks!
Mark
Other great posts!
P.S. I also found these posts to pose great questions and/or different (and valuable) perspectives:
Improving the newbie warnings on “low-quality” answers (2 Sep 2016)
Extraordinary spike in low-quality questions by 1 rep users (10 Jan 2016)
Am I wearing rose-colored glasses?
P.P.S. I acknowledge that I might be overly optimistic about conducting such testing, given that similar research seems stalled...
Let's Plan the Second Iteration of the Stack Exchange Quality Project! by @Tim Post on 17 Oct 2016
We plan to test a new, 'guided' version of the ask question page soon. This page would essentially break down all of the elements that make a great question, and give the user plenty of guidance as to why it's to their advantage to understand what's needed in each of them, and provide it.
I'll be announcing some tests within the next 3 or so weeks, as well as a survey that's shown to brand new users after they ask their first question. tl;dr; - we think we know how to optimize the ask page to get good results, but we want to confirm that we're correct here - or a really good idea might get shelved due to a series of poorly constructed tests. – Tim Post♦ Oct 25 '16 at 19:04
Did anything come of those tests, @TimPost? – Josh Caswell Jun 10 at 16:29
(no response as of 20 Jul 2017)
When will we hear about the outcome of this initiative/? – Raphael Feb 5 at 19:29
@Raphael 6-8 months/years. (aka the team is busy with other things.) – Shadow Wizard Jul 4 at 9:04
@ShadowWizard As in, we all wasted our time (it may never happen, or things may be obsolete when it happens). Way to go! Next time, if you ask for several man weeks (?) of community feedback, please make plan beforehand on how you'll close the loop. Thanks. – Raphael Jul 4 at 9:17
Nobody wasted time here. More announcements coming this week as things move forward. – Tim Post♦ Jul 4 at 12:12
@ShadowWizard maybe it's just not yet time to take a stand on these matters – gnat Jul 5 at 6:06
@Tim with all due respect (and there is respect, lots of it actually :)) the people who posted here wasted their time, and that's a fact. Proof? Let's take the top answer here, with 399 upvotes, posted almost year ago. You did post a promising comment, but those 3 weeks you mentioned turned into over 8 months now. So Robert has the full right to feel he wasted his time. I do not blame you. I do not blame anyone. Just stating the facts here. – Shadow Wizard Jul 5 at 6:16
