If you work out with me, you'll quickly tire.
If you talk with my voice, you've lost your desire.
When a joke falls to me, I make nobody laugh.
When I annotate a letter, I take away half.
Who or what am I?
In your answer, please explain each line and the title.