I've seen over the past couple days a couple of questions that I think bring up an issue we really need to discuss: how to handle the X vs Y questions.

I think we can all agree (even the poster agreed!) that we shouldn't simply have X vs Y questions. Consider QTP vs Selenium.

But others are not so clear cut.

Where do we draw the line between the first one and the second set (if the second set is to be allowed, that is.)

Are we planning to take the super user approach of "no product recommendations"? Do we plan to limit it to just specific feature sets and say "this product has this feature, that product has that feature. Your list of requirements are features X Y and Z, so the product I can find that meets the most of those is blah."

tl;dr How can we define an objective set of guidelines for what constitutes an appropriate X vs Y question?


To kick off discussion on this one...

I'd say that anything in the form of "Is tool X better than tool Y?" is not appropriate.

Questions in the form of "Is tool X or tool Y better for task A?" is borderline - it will depend on what the task actually is, and perhaps a request for more detail would be the best approach.

Questions in the form of "Is tool X or tool Y better for task A in envronment Q with parameters Tic, Tac, and Toe?" give enough information to be able to provide an objective response - although it's possible that the objective response could be "Either tool will do be fine for this" or even "Neither of those tools can do this."

  • +1 I think this is a decent place to start, at the very least.
    – corsiKa Mod
    May 27 '11 at 5:08
  • Agreed. I'd say we want to be encouraging people towards the third form ideally.
    – testerab
    May 29 '11 at 16:38

Would it help to point people towards some hints on how to structure a good question? I've put together a start here, I'm making this community wiki so it can be edited.

Good "X vs Y" questions put the problem in the foreground, not X or Y. Without that information, answers are likely to be based purely on personal preference/popularity - if you explain what you're trying to do, this allows people to address their answers to the specific features of X or Y that are relevant to solving your problem. They will be able to suggest alternative solutions that you might not have considered.

If you're having difficulty in writing your question, you might find it helpful to try putting it into the following format:

  1. Problem statement: What problem are you trying to solve?
  2. What's your context (see below for more detail on what this might cover)? Any other constraints?
  3. What features of X do you think may be particularly relevant to your problem? What features of X are potentially problematic?

Context and constraints:

What's the overall environment, and how does this potentially constrain your choices? Depending on the question, this could include details about operating systems that would need to be supported, budgetary constraints, development methodology, industry/regulatory requirements, and so on. E.g.

  • "We follow a Prince2 delivery process, and the test team and development team are on different sites."
  • "The company I work for requires that all tools are approved before use: due to timescales and internal politics, this means I can only select tools that are already on the list".
  • "My manager says I can pick any tool I want, so long as it's free."
  • "The solution we pick will have to be suitable for non-technical business analysts too..."
  • "We currently use tools A, B, and C - changing those isn't an option for us."

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .