Find all functions $f:\mathbb{R}\rightarrow\mathbb{R}$ such that $$f(x)f\big(f(x)+y\big)=f\big(x^2\big)+f(xy)$$ for all $x,y\in\mathbb R$
Problem by me
Most elegant solution wins!
Not sure this rates as elegant but it is certainly short:
At $x=0,y=-f(0)$ we have $f(0)^2=2f(0)$, a quadratic equation with solutions $0,2$.
Case 1: $f(0)\ne 0$
substitute $x=0$ to get $2f(y+2)=4$ for any $y$, therefore $f\equiv 2$.
Case 2: $f(0)=0$
Case 2a: there is another zero $f(z)=0,z\ne 0$
substitute $x=z$ to get $f(z^2)=f(zy)$ for any $y$, i.e. $f$ must be constant $f\equiv 0$.
Case 2b: $f(0)=0$ is the only zero
Then, letting $y$ equal $0$ and $-f(x)$, respectively, we can make good use of $$ f(x)f(f(x))=f(x^2)=-f(-xf(x)) $$ With $x=1$ this yields $f(f(1))=1$ and $f(-f(1))=-f(1)$. With $x=f(1)$ we now get $f(f(1)^2)=f(1)$ and with $x=-f(1)$, finally, $f(1)=1$ and $f(-1)=-1$.
We now go back to the original equation. Comparing $x=1,y=t$ with $x=-1,y=-t$ we get $f(-z)=-f(z)$ for any $z$. Finally, setting $y=-x$ we get $f(f(x)-x)=0$, i.e. $f(x)=x$ for any $x$.
First, let's get rid of all constant solutions. If $f(x)=c$ is a solution, then the equation gives
$c^2=2c\implies c\in \{0,2\}$. So from now on let's assume $f$ is non-constant.
If some $x$ satisfies $f(x)=0$, then plugging that in,
$f(xy)=-f(x^2)$, and since $f$ is non-constant, $x$ must be zero. Also, putting $x=0$ gives $f(0)f(f(0)+y)=2f(0)$, and again $f(0)\ne 0$ leads to $f$ being constant, so in fact $f(x)=0\iff x=0$.
Plugging $y=0$ now gives
$f(x^2)=f(x)f(f(x))$, so the equation can be written $f(x)f(f(x)+y)=f(x)f(f(x))+f(xy)$. Call this alternate version $(\star)$.
Now we claim that
$f$ is injective. Indeed, suppose $f(a)=f(b)$ with $a\ne b$. As seen already, $a,b$ are non-zero. Now substituting $x\mapsto a$ and $x\mapsto b$ in $(\star)$, we see that all the terms except $f(xy)$ have the same value in both cases. Thus $f(ay)=f(by)$ for all reals $y$. This shows that $f$ takes the same value on any two reals with ratio $a/b$.
Now plugging $x=1$ in the original equation gives $f(1)f(f(1)+y)=f(1)+f(y)$. Using this equation for $y\mapsto ay$ and $y\mapsto by$, we see that in fact $f(f(1)+ay)=f(f(1)+by)$, and given any $c\ne a/b$, one can choose $y$ so that the numbers $f(1)+ay$ and $f(1)+by$ have ratio $c$. Thus given any ratio, one can find two reals with that ratio so that $f$ has the same value on them.
And thus, to prove injectivity,
Note that given any non-zero $x_1,x_2$, we can find $c,d$ so that $c/d=x_1/x_2$ and $f(c)=f(d)$. Using the same kind of argument as on $a,b$, this would give $f(cy)=f(dy)$, so in particular $f(x_1)=f(x_2)$, so in fact $f$ is constant on non-zero reals. Because of $f(x^2)=f(x)f(f(x))$, the only possibility (remembering $f$ is non-constant) is $f(x)=1$ for non-zero $x$ and $0$ for $x=0$; this function, however, doesn't work! So $f$ has to be injective.
Phew! Now we are ready for the finish. Indeed, plug in
$y=-f(x)$; we get $f(x^2)=-f(-xf(x))$. Putting $x\mapsto -x$ gives $f(-xf(x))=f(xf(-x))$, and by injectivity this gives $f(-x)=-f(x)$ (well, as long as $x\ne 0$, but this holds for $x=0$ anyway so we are good). This says $f$ is an odd function, so in fact $f(x^2)=f(xf(x))$. Again, injectivity allows us to "cancel out" the outer $f$'s, giving $f(x)=x$ (again, for non-zero $x$, but $f(0)=0$ is already known).
Thus these are all such $f$:
The constant functions $0$ and $2$, and the identity function.
Not an answer, but a step toward an answer:
Let $x$ be a number such that $f(x)=0$. Then the definition becomes $0=f(x^2)+f(xy)$. Thus $f(xy)$ is constant as a function of $y$ (for this $x$) and thus either $x=0$ or $f$ is constant. We wind up with two choices: $f\equiv 0$; or $f(x)=0\Rightarrow x=0$ (which latter includes the case that $f$ is never $0$).
(The examples already found by other answerers —
$f\equiv 0$, $f\equiv 2$, and $f(x)=x$
— all meet this criterion.)
I'm not sure about "all functions", but I've found 2:
$f(x) = 0$
and
$f(x) = 2$
Along with the solutions provided by Steve, I've found that:
$f(x) = x$
Is also a valid solution
$f(x)f(f(x)+y) = f(x)f(x+y) = x(x+y) = x^2+xy$
$f(x^2)+f(xy) = x^2+xy$ (Note this doesn't apply for $f(x) = cx$ where $c\ne1$, as you result with $c^3x^2 + c^2xy = cx^2+cxy$