What should you look for when rating the difficulty of a puzzle?

Question

After you're finished making a puzzle, what types of details should you look for when rating it's difficulty? What makes a puzzle easier or harder? Keep in mind a puzzle includes both the question and the answer.

Answer 1

The best way to run difficulty tests on your puzzle is the tried-and-true method of a test audience. As others have pointed out, do not include yourself in this audience - you are biased because you already know the solution.

You can try to be rigorous in your testing b getting a large sample audience to test your puzzle against, which is the usual manner in which these things are tested (analogous to Game Testing), but for smaller puzzles, or if you're strapped for participants, you can choose a small handful.

If this is a personally-made puzzle for fun, you're more likely to use a small handful of participants, so choose your testers wisely. Know how skilled they are at puzzles of your type ahead of time, and base their test results on that information - if they're not typically good at puzzles, them solving it quickly might be a sign that you need to make it harder, though if they're particularly good at it, taking a very long time when they shouldn't have a problem could be a sign that you need to make it easier or include more hints.

Testing puzzles is an imprecise science. Sometimes what seems like a minor adjustment can make the puzzle unbeatable, and other times your focus on a preconceived solution will blind you to an obvious 'easy' solution you didn't think about before.

And of course this depends on what type of puzzle you've made - if it's wholly new, you'll need a lot more testers to really get a feel for how hard/easy you've made it.

Answer 2

Your puzzle is probably harder than you think it is.

This is just a heuristic answer. Some puzzles turn out easier than the writer expects. But in my experience, it's much more common error to make a puzzle that's harder than you intend.

When judging the difficulty of your own puzzle, it's easy to forget that you have the advantage of knowing the answer. It sounds silly, but it's hard to push what you know out of mind and take an outside perspective.

You've had this puzzle in your head for a while, it just seems so natural to you. And a puzzle that you make is naturally of your style, which makes you better at solving it than most.

It's tempting to think a puzzle is easy because the steps are easy. But to a solver, the correct steps and deductions and just one possibility of many. They might have to try a bunch of steps that don't work. And sometimes, they try one that does work, but still continue to try other ideas, or even discard the right one because they don't realize another step is needed for it to make sense.

Any interpretation puzzle is liable to have coincidental patterns, things that seems to kind-of fit. Confirmation bias is hard to shake, and solvers might get stuck on wrong theories. And that's fine, as long the right answer is clearly better, but you need to account for these detours in assessing difficulty.

Lastly, flaws in design tend to make a puzzle harder. Unintentional errors, misworded clues, unintuitive conventions. Of course, ideally a puzzle shouldn't have these, but nobody is perfect, so consider erring on the easier side to account for this. Cluing things multiple ways (redundancy) is a good way to make a puzzle resilient against these faults.

Answer 3

You didn't say what kind of puzzles you were asking about. I enjoy programs that generate logic puzzles. I suspect for those, the programmer writes a solver with whatever logic s/he thinks the person solving should use. The program then generates a solution and feeds the (empty set) of clues to the solver. If the solver fails, you add a clue and try again. For Sudoku, there is an agreed hierarchy of types of reasoning. The difficulty assessment is what level of reasoning is needed. The solver can have branches to avoid certain levels of reasoning, and add more clues as required.