The answer is
The chances are equal
Proof
Suppose we consider all the possible outcomes. Then I propose, there is a one-to-one mapping from winning outcomes for you to winning outcomes for me.
To see this, consider a scenario where you roll $i$ and I roll $j$ and you win ($i > j$). Then, in the case, that you roll $-i$ and I roll $-j$, I win (since ($-i < -j$). The mapping $(i,j) \rightarrow (-i, -j)$ is clearly 1-1 for the given problem, it inverts the winner, and the set of draws maps onto itself.
If we were to enumerate overall all possible outcomes, I would win the same number of times as you so the probability of winning must be the same.
NB: This additionally show that the chances would be equal if we replaced $5$ and $20$ by any integer values.
Alternative proof
The probability of winning with the $41$-sided dice in a single go can be summed as the probability of winning given that the $11$-sided die rolls a $5,4,3,\ldots$ multiplied by the probability of rolling a value in an $11$-sided die that is $$ P(41\text{ wins}) = \frac{1}{11}\displaystyle \sum_{j=15}^{25} \frac{j}{41} = \frac{220}{451}$$ Similarly, the probability of the $41$-sided die losing is given by calculating a similar sum $$ P(41 \text{ loses}) = \frac{1}{11}\displaystyle \sum_{j=25}^{15} \frac{j}{41} = \frac{220}{451}$$ where the sum is descending over the integers in this case. The probability of a single outcome being a draw is just $\frac{11}{451}$.