A generalization to arbitrary centrally symmetric regions
A planar region is called centrally symmetric with respect to the origin, if for every point $P$ in the region also its reflection with respect to the origin is in the region.
Examples for centrally symmetric regions are for instance circles, squares, regular hexagons, regular polygons with an even number of sides.
Theorem:
Let $R$ be a (bounded) region that is centrally symmetric with respect to the origin.
If three points are randomly chosen from $R$, then the probability that the resulting triangle contains the origin is $1/4$.
As a special case, we derive that the corresponding probability for a circle is also $1/4$.
-----------------------------------------------------------------------------
The proof
(1) My argument heavily uses the fact that exactly one of the following two statements holds true for any triangle $\Delta$:
- The triangle $\Delta$ does contain the origin.
- There exists a line that separates the origin from the triangle $\Delta$.
(2) Now consider the region $R$, and let $\ell$ be a horizontal line through the origin.
We will assume throughout that none of the three randomly chosen points does ever land directly on line $\ell$ (as these cases have measure zero in the underlying probability space).
A situation where the triangle contains the origin is called "success", and
a situation where the triangle does not contain the origin is called "failure".
(3) We distinguish three cases for three randomly chosen points $A,B,C$ in the region $R$:
Case (3a): All three points are above $\ell$.
Case (3b): All three points are below $\ell$.
Case (3c): Two points are above $\ell$, and one is below $\ell$.
Case (3d): Two points are below $\ell$, and one is above $\ell$.
The respective probabilities for cases (3a), (3b), (3c), (3d) are $1/8$, $1/8$, $3/8$ and $3/8$. Furthermore, cases (3a) and (3b) are clear failures (as line $\ell$ itself separates the origin from the resulting triangle). By symmetry, the success probabilities for cases (3c) and (3d) are identical and will be called $p^*$.
(4) Now let us compute the success probability $p^*$ for case (3c). Assume that two points $A$ and $B$ are above $\ell$, whereas the third point $C$ is below $\ell$.
The triangle $\Delta ABC$ is a success, if and only if the line through $C$ and the origin has $A$ on one side and $B$ on the other side.
Let point $C'$ be the mirror image of $C$ with respect to the origin.
The triangle $\Delta ABC$ is a success,
- if and only if the line through $C'$ and the origin has $A$ on one side and $B$ on the other side,
- which happens if and only if in the clockwise ordering of $A$, $B$, $C'$ around the origin we have point $C'$ in the middle between $A$ and $B$.
Now note that the probability distribution for choosing $C$ is identical to the probability distributions for choosing $C'$, for choosing $A$ and for choosing $B$, as all these points are chosen from congruent regions (that are one half of the symmetri region $R$).
Consider an arbitrary choice of three points $X$, $Y$, $Z$ from the half-region. If we randomly label these three points by $A$, $B$, $C'$, then exactly two of the six labelings have $C'$ in the middle between $A$ and $B$.
Summarizing, the success probability in case (3c) is $p^*=2/6=1/3$.
(5) Now let us wrap things up. The overall success probability equals the overall success probability in the four cases
Case (3a): $\frac14\cdot0$
Case (3b): $\frac14\cdot0$
Case (3c): $\frac38\cdot p^*$
Case (3d): $\frac38\cdot p^*$
As the sum of these four success probabilities is $\frac34\cdot p^* =\frac14$,
the proof is complete.