You are:
BACKWARDS!
My beginning, as you may see,
has a unit of account.
That it does take first parts for me
is clearly paramount.
Original answer: In finance, a DRAW is a transaction involving the movement of money from an account (hence 'withdraw' or 'overdrawn').
However: as pointed out in comments the OP's intention was SDR (Special Drawing Rights) which according to Wikipedia are "supplementary foreign exchange reserve assets defined and maintained by the International Monetary Fund. SDRs are units of account for the IMF, and not a currency per se."
My middle, for writing refined,
is a symbol for its correction.
If you look, then you may just find
something unwieldy on inspection.
The abbreviation AWK is used when marking essays or a transcript, to signify a sentence that does not flow smoothly (i.e. is 'unwieldy').
My ending, maybe drawn by horses,
carries people to and fro.
Travelling all the city courses,
its color now, is yellow.
This is indeed a CAB, as discovered by @trolley813 - something traditionally drawn by horses but famously yellow in the context of New York's taxicabs.
My whole is returning -
the solution's in hand.
Are you quite discerning?
Then you may understand.
Altogether, SDR + AWK + CAB spells SDRAWKCAB - but in reverse(!) ('returning') this spells (appropriately) BACKWARDS!