As IT professionals all, we know that metrics are metrics are metrics and that they can be twisted any which way. But they can also be useful. We have 5 primary metrics by which our site is publicly judged:
Questions per day - current 2.9 and in the 'needs work' category
Questions answered to total questions ratio - currently 89% and in the 'okay' category
Avid users - currently 156 and in the 'excellent' category
Answer ratio - currently 2.9 and in the 'excellent' category
Visits per day - currently '3,667' and in the 'excellent' category
Now, there's clearly two there that need work. One is questions per day which, unless we somehow have a host of QA problems we just need solved right away, we can't make go up on our own. We could ask problems that we're thinking about, but those at best make mediocre questions: the best questions are based off of actual experience and problems we're facing right now. (I've also found a trend that you get about 1 question per 1,000 views. So if we want more questions, we really need to up our daily visits. But that's another topic.)
The other one is the ratio of answered questions to total questions. And this is something I think we have a lot more control over. There are currently 189 unanswered questions. Usually, questions are unanswered for one of three reasons.
- They're bad questions and should be closed.
- They require more context to be accurately answered.
- They're difficult questions and need experts.
Now, if only 10 people handled a single question on their lunch break, we'd be done well before the end of March. I think we can get this metric down a fair bit. So let's try the following:
- Each day, we all find a single question in the Unanswered queue to handle.
- If it has a good answer that doesn't have the upvotes it deserves, Click Upvote.
- If you can answer the question, do it,
- If the question is a bad question, vote to close it.
- If the question needs more context and is more than 6 months old, vote to close it.
- If the question needs more context and is less than 6 months old, ask for the necessary clarification.
- If someone has asked for clarification as part of this drive, but no response has been given in 7 days, vote to close it.
- If none of the above apply and you can't answer it, share it with someone who you think can.
- If you aren't sure because it's an edge case, bring it to meta. You don't need to write a novel in your meta post. Just link the post, and explain your problem in a sentence or two.
- Keep an eye on the close queue so the old questions that people have identified get cleaned up quickly.
- Keep an eye on meta so we can resolve questions about edge cases.
Most sites that we might compare to activity wise have a 98-100% answered ratio.
I challenge sqa.se to get our answered ratio to 97% by the end of March.
We kind of talked about doing this a year ago. Having a higher answered ratio increases user confidence and creates a culture of completeness. It's also something we can actually do. We can't make others come to the site. We can advertise, we can tweet, and the like, but we can't actually make them click the site. But we can raise the quality of the site they click on.
Update March 11th: We're currently 1/3 of the way through, and we've gone up between 1 and 2%. We've officially crossed into the "Excellent" category, which is good, but we still have some cleanup to do. We're down to 167 questions, which puts us on pace to be between 120 and 125 at the end of march, or at 93%. So if we're going to meet the challenge of 97%, we still have a ways to go, but I think we're making great progress.
Something I'm seeing a lot of is questions that have answers with no votes. If a question has 4 answers, chances are at least one of them is worth an up-vote. If four different people attempted to answer a question and were unable to get anything useful, it probably means the question is a bad one, so give it a vote-to-close, flag it, bring it to meta, etc.
Well we did pretty good. Did we get to 97%? Well, no. But that's not the end of the world. We did, however, push us very far into the 'excellent' category. There's still some great hidden gems in there, both in questions that could use some good answers, and some good answers that could use the votes they deserve.
Let's not forget that we also are fighting existing questions. Some days we went up in the number of unanswered questions by 4 or 5. So on those days, even if it didn't look like we made progress, we certainly did.
Thanks for a great March Madness, folks =D