My guess:
An n-dimensional sphere.
I'm very common and often you see me,
There are lots of spheres
Everything's believed to be made of me.
This could refer to the https://en.wikipedia.org/wiki/Bohr_model of atoms...
Make no mistake, I look largest when I'm seven,
But I'm largest when I'm five, it is proven.
That was one of the keys:
As the diagram at https://en.wikipedia.org/wiki/Hypersphere shows: The surface of the sphere is largest in dimension 7. But the volume reaches its peak at dimension 5.
But alas at those ages you've never seen me,
For you've seen me only when I'm three.
We can see a sphere only (or at most) in 3 dimensions
Unlike you mortals who grow old and die,
I keep shrinking and shriveling by and by,
I don't die when I'm seventy, eighty or ninety,
I only kneel and perish after the infinity.
The surface and the volume of an n-dimensional sphere approach zero when n goes towards infinity. Even at 70 or 80 dimensions, there's hardly anything left.