It rains in Gadyukino village every other day on average. Assume the probability of rain on any given day is 50% and independent. A local weather forecaster has the sole duty every day of predicting whether it will rain the following day. His predictions have a 75% success rate. What quantity of information (in bits) is carried by a single forecast of his?

  • $\begingroup$ In my view, it is reasonable to try interpreting the problem assuming the statement is complete and correct - and the initial statement allowed such an interpretation, including independence of days (otherwise the statement would be incomplete about how the history affects the weather), and also about the quantity (how much) of information (see quantities of information). $\endgroup$ – user3900460 Mar 22 '15 at 18:17

"How much information" sounds vague. But it has a rock-solid definition, in the grand tradition of Shannon et al.

Assume an independent 1/2 chance of rain every day. Then the weather transmits one bit of information to Gadyukino every day - namely, whether or not it is raining. Say the forecaster can receive $x$ bits of information per day about the future. Then what is the minimum value of $x$ so that in the limit, the forecaster predicts with 75% accuracy?

Consider a time period of $4n$ days for large $n$. There are $2^{4n}$ possible weathers during this period. For any particular forecaster prediction sequence, the number of weather sequences that let the forecaster claim a 75% accuracy rate is $\binom{4n}{\leq n}$, where the notation means the number of ways to choose at most $n$ wrong days out of $4n$ total days.

Stirling's approximation says that $\binom{4n}{\leq n} = 2^{(4\log 4 - 3 \log 3)n}\times poly(n)$. Call this quantity $X$ for ease of typing.

Each forecaster prediction sequence allows $X$ possible weather sequences. There are $2^{4n}$ total weather sequences. Therefore, the forecaster must have at least $2^{4n}/X = 2^{(4 - 4 \log 4 + 3 \log 3)n}$ possible prediction sequences that he might say.

In order to be able to choose from that many prediction sequences, the forecaster must receive at least $\log(\text{that many})=(4 - 4 \log 4 + 3 \log 3)n$ bits for those $4n$ days. (EDIT: This doesn't account for the fact that the forecaster can change his mind based on whether he's been right so far. My intuition says an information-theoretic bound should still work, I'll think about it.)

This works out to $\boxed{1-\log 4 + \frac 34 \log 3 \approx 0.43766...}$ bits per day. This is an upper bound on the amount of information the forecaster needs. Does this much information suffice? I don't know. Shannon probably solved this problem when he created information theory, but I'm having trouble finding a reference.

| improve this answer | |

It rains on alternate days on average, which means that:

  • If today it's sunny, it's likely that it will rain tomorrow.
  • If today it's raining, it's likely that it's sunny tomorrow.

The forecasters peruse the recent weather statistics and discover that 75% of the sunny days were followed by a rainy day. So, if today it's sunny, we have a 75% probability of rain tomorrow. Why we assumed that today it's sunny? Because it says that rains occurs every other day, and if today it's rainy, a 75% chance of rain for tomorrow would conflict with the assumptions.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.