7

As IT professionals all, we know that metrics are metrics are metrics and that they can be twisted any which way. But they can also be useful. We have 5 primary metrics by which our site is publicly judged:

Questions per day - current 2.9 and in the 'needs work' category

Questions answered to total questions ratio - currently 89% and in the 'okay' category

Avid users - currently 156 and in the 'excellent' category

Answer ratio - currently 2.9 and in the 'excellent' category

Visits per day - currently '3,667' and in the 'excellent' category

Now, there's clearly two there that need work. One is questions per day which, unless we somehow have a host of QA problems we just need solved right away, we can't make go up on our own. We could ask problems that we're thinking about, but those at best make mediocre questions: the best questions are based off of actual experience and problems we're facing right now. (I've also found a trend that you get about 1 question per 1,000 views. So if we want more questions, we really need to up our daily visits. But that's another topic.)

The other one is the ratio of answered questions to total questions. And this is something I think we have a lot more control over. There are currently 189 unanswered questions. Usually, questions are unanswered for one of three reasons.

  1. They're bad questions and should be closed.
  2. They require more context to be accurately answered.
  3. They're difficult questions and need experts.

Now, if only 10 people handled a single question on their lunch break, we'd be done well before the end of March. I think we can get this metric down a fair bit. So let's try the following:

  • Each day, we all find a single question in the Unanswered queue to handle.
    • If it has a good answer that doesn't have the upvotes it deserves, Click Upvote.
    • If you can answer the question, do it,
    • If the question is a bad question, vote to close it.
    • If the question needs more context and is more than 6 months old, vote to close it.
    • If the question needs more context and is less than 6 months old, ask for the necessary clarification.
      • If someone has asked for clarification as part of this drive, but no response has been given in 7 days, vote to close it.
    • If none of the above apply and you can't answer it, share it with someone who you think can.
    • If you aren't sure because it's an edge case, bring it to meta. You don't need to write a novel in your meta post. Just link the post, and explain your problem in a sentence or two.
  • Keep an eye on the close queue so the old questions that people have identified get cleaned up quickly.
  • Keep an eye on meta so we can resolve questions about edge cases.

Most sites that we might compare to activity wise have a 98-100% answered ratio.

I challenge sqa.se to get our answered ratio to 97% by the end of March.

We kind of talked about doing this a year ago. Having a higher answered ratio increases user confidence and creates a culture of completeness. It's also something we can actually do. We can't make others come to the site. We can advertise, we can tweet, and the like, but we can't actually make them click the site. But we can raise the quality of the site they click on.


Update March 11th: We're currently 1/3 of the way through, and we've gone up between 1 and 2%. We've officially crossed into the "Excellent" category, which is good, but we still have some cleanup to do. We're down to 167 questions, which puts us on pace to be between 120 and 125 at the end of march, or at 93%. So if we're going to meet the challenge of 97%, we still have a ways to go, but I think we're making great progress.

Something I'm seeing a lot of is questions that have answers with no votes. If a question has 4 answers, chances are at least one of them is worth an up-vote. If four different people attempted to answer a question and were unable to get anything useful, it probably means the question is a bad one, so give it a vote-to-close, flag it, bring it to meta, etc.


Well we did pretty good. Did we get to 97%? Well, no. But that's not the end of the world. We did, however, push us very far into the 'excellent' category. There's still some great hidden gems in there, both in questions that could use some good answers, and some good answers that could use the votes they deserve.

Let's not forget that we also are fighting existing questions. Some days we went up in the number of unanswered questions by 4 or 5. So on those days, even if it didn't look like we made progress, we certainly did.

Thanks for a great March Madness, folks =D

7
  • I've started actively digging through the unanswered questions queue. There's some good ones buried in there. – Kate Paulk Mar 5 '14 at 12:12
  • 1
    I ran across several unanswered questions of the form, "Is there a tool that does X?" Such a tool may exist somewhere in the universe, and so it would be improper to answer "No" and unproductive to answer "Not that I know of". What should we do with those? – user246 Mar 5 '14 at 13:01
  • 3
    @user246 I would VTC that as 'Too Broad'. Questions should be focused on problems, and leave the solutions for the answers. "Is there a tool that does X" is an inferior question to "I want to X but I can't because of Y". – corsiKa Mar 5 '14 at 15:42
  • @corsiKa. Let's test that on an example. If someone asks whether there's a Jenkins plugin for displaying performance data from JMeter, would you say that question is too broad? – user246 Mar 6 '14 at 16:31
  • 1
    I've added some answers, but without any upvotes they remain on the unanswered list – Yamikuronue Mar 14 '14 at 15:43
  • That's excellent. Because there's activity on that question it should bubble to the top of the list, so people should see them and provide some feedback. Thanks for assisting with this project! – corsiKa Mar 14 '14 at 17:44
  • ok, so I went through the really old ones and some of those the answers are in the comments, but it clutters the "unanswered" query. Also it seems people in the SQA group don't like to "accept" answers like in standard stack overflow...can we get the "unanswered" to really reflect correctly unanswered instead of things where comments are answered or they got an answer but didn't click the accept answer button? – mutt Mar 25 '17 at 19:12
2

Imho and ime it is early days here. No harm in leaving unanswered questions if they cannot yet be answered.

The #1 priority here should be ways to increase participation.

Some of the targets proposed above will reduce the possibilities for participation. There are questions closed as 'too broad'. Leaving questions open to allow new members to comment or answer will increase participation.

A beginning answer for many of these would have increased the knowledge of the person asking it, and that is what we are all here for. The quality will increase in time.

Yes some of them are posted by persons just beginning their study and practice. If you want these persons to return and contribute when they become more knowledgeable then please begin the relationship by making a simple contribution to advancing the question even if it is only a few lines in the right direction.

2
  • If a recently posted question is too broad, then as a community I agree that we should be encouraging users to add enough information that it's no longer too broad, by commenting and prompting, or editing ourselves when it's possible to do so (sometimes it really isn't possible to figure out). But for an old question, especially where people did that and got no response - clear it out! Old bad questions discourage participation by making it harder to find useful information. – testerab Mar 9 '14 at 12:54
  • 3
    I can see that, for sure. But chances are after 3-6 months, a user isn't coming back. One thing I do, though, is to look at who posted the question. If the user is someone who is active on another Stack Exchange site, they will get an @ notification. I just pinged a guy who hadn't been on the site in nearly a year, but had been active on Stack Overflow within the past couple days. So I commented and he replied (he had not solved the problem). I'm okay leaving that open knowing it would be attended to if someone replied. – corsiKa Mar 10 '14 at 15:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .