My answer is:
Wonder
As a young child, you live here.
All children, especially when very young, find most of the world wonderful
Later you leave me often.
As you grow up, you get sometimes frustrated, realist
But you always return, sooner or later.
Some things still marvel you, even when you're an adult
I have no beginning or end. I am infinite.
Not quite sure, but this may only mean "what is to be guessed is abstract"
Therefore, if God exists,
...Which I surely WONDER
God must live here, too.
God would have all the qualities of a wonderful being, therefore living in wonder