# How to rate difficulty of a classic riddle?

How, as a writer of riddles, can I rate their difficulty?

I guess, for me alone, knowing the answer even before the riddle existed, it's pretty much impossible, and I should need some external help. Are there some riddling communities or something like that, who could help rate the difficulty? How would I approach this problem?

Edit: to clarify. I mean classic riddles based on metaphors - like the ones found in Tolkien's books. Not 'trick questions', math problems posed as riddles, or other weird riddle-like creatures.

• I've voted to close this question, based on the criterion here. – user20 May 17 '14 at 20:26
• This is a bit unclear... are you looking for a system for grading riddle difficulty? – jimsug May 17 '14 at 23:42
• @jimsug: If such a system exists, I'd love to hear about it, but if it doesn't, I'd like to at least know if my riddle is so easy everyone has the answer two verses in, or too hard, with nobody ever able to find the answer. Unfortunately, as the span of riddle difficulties goes, the two are more likely to occur than just a riddle that requires a deep thought over the clues. – SF. May 18 '14 at 15:16

I think this is very hard question to answer practically. So even though I do not have an direct answer to it I would like to share my experience, which in related area.

I am living on a Russian site, that provides unique puzzles for people to solve. Here is the facts about how hard is to estimate puzzle hardness for us:

1. It has about 400 selected puzzles and 55000 users, 1000 from which are active.
2. We have a rating (puzzle hardness) system, which calculate a hardness of a puzzle basing on ratio $\frac{NPeopleSolved}{NPeopleSeen}$, where $NPeopleSeen$ is number of people, which solved at least one arbitrary puzzle after the given puzzle was published.
3. Based on 7 years experience we decided to have only 5 possible hardness categories for puzzles. That means we trying to estimate the hardness with precision of 20% only. (The hardness category depends exponentially on the ratio above, if you are interested).
4. Again, currently we decide that we need at least 1 month to give first estimation of the hardness. That means that first of all a puzzle is published on a site and is solved by people during one month, only then we set a hardness for it.
5. In average (taking into account all period the puzzle is exists on site) we have about 100 people who solve the puzzle per months for simplest puzzles and only 1 guy per months for hardest. Fortunately for first months this ratio is bigger due to the fact that active people are clever in average and hungry for new puzzles.
6. Buy the fact the hardness is not perfectly stable even for puzzles, which have collected years of solving-statictics. Due to changes in site popularity, kind of people attracted and allowed to the site it can change with time. So it clearly depends a lot on selection of people to estimate the hardness.
7. Before the current automatic system described above we had a set of moderators, who tried to set the puzzle hardness by voting. And the results of statistics and voting appeared to be quite different when compared. Similarly I (even having experience of solving puzzle from beginning to the end) can rarely predict the hardness before the puzzle is published, I am surprised often how very simple puzzles for me can be hard for average user and how very hard puzzles for me can have the lowest hardness on the site.

I never heard about rating community and even it is exists it hard to imagine how it can estimate the hardness of the puzzle more or less correctly and meanwhile cheap enough (in people*hours) and quick enough. Especially if your puzzles is hard enough for clever person to think about it for whole month.

• Actually, what you described is exactly the kind of community I had on my mind; solve puzzles, and difficulty gets rated through statistics of success rate. – SF. Feb 2 '15 at 19:21

I think the problem with trying to rate these with one number is that most of the most popular brainteasers present challenges on up to three different "challenge spectrums":

• Some are highly dependent on actual skills, knowledge or tools, like ones that can't easily be solved without a spreadsheet, or that are incredibly hard without knowledge of mathematical techniques like probability (trees, permutation odds, etc.) Monty Hall is one example that is much harder to solve -even if you can get past how unintuitive the answer is -if you've never studied at least basic probability techniques.

• Some require complex reasoning or creative (but not "sneaky") thinking. Complex reasoning (including some experimentation with different approaches) would include puzzles like 100 boxes. the I'd toss out "Everyone may have children until they have a boy, and then must stop - what will the eventual boy/girl ratio be?" as an example of creative thinking - it's not a "trick," but you have to approach it in a highly non-intuitive way.

• Some require lateral thinking, of the variety that can seem kind of like a trick. I've seen this referred to as the "Aha!" characteristic of a puzzle. Classic examples end with "The doctor is his mother (you sexist!) or "The victim is a fish!!"

I would say that riddle difficulty is hard to rate. Some people will find one riddle easy, while another person will find it hard. It all depends how you approach the solution. Some riddles involve little-known facts, and therefore are difficult to solve.

Ask a bunch of people this riddle. Then, rate the difficulty depending on how many people you ask before someone gets it. If it takes 6 people, rate the difficulty 6.