My answer is:

>!Wonder

As a young child, you live here.

>!All children, especially when very young, find most of the world wonderful

Later you leave me often.

>!As you grow up, you get sometimes frustrated, realist

But you always return, sooner or later.

>!Some things still marvel you, even when you're an adult

I have no beginning or end.
I am infinite.

>!Not quite sure, but this may only mean "what is to be guessed is abstract"

Therefore, if God exists,

>!...Which I surely WONDER

God must live here, too.

>!God would have all the qualities of a wonderful being, therefore living in wonder


Edit: I actually had another solution, which was 

>!Phantasy