Word. As noted by others, the first letter of the lines spell LOGOS,which in ancient Greek means a lot of things, but primarily "word".
Look towards me, I was there at the start,
The prologue of the Gospel of John says: "In the beginning was the Word, and the Word was with God, and the Word was God".
Open your mouth, I represent your heart.
The words we say represent our heart.
Go towards the voice, it is me who you'll find,
If you follow a voice you're probably following words.
Of covenants and deals, I can be used to bind.
"To give one's word" means to bind a contract with ones honour.
Sought for by those, who are seeking the truth.
The Word as in the Christian sense, I think.