My answer is:
your eyelids.
I am that which you see the least,
Most of the time when we are actually seeing things, our eyes are open, so we don't see our eyelids. Closing our eyes sorta lets us do it, but barely so.
and with what you see, does relate.
We see with our eyes and eyelids are definitely related to our eyes.
No matter how inseparable we are
you’ll always miss me.
Eyelids are obviously on us and are so close to that which we see with, but as soon as we try to bring them into our field of view (blinking/closing our eyes), we obscure our ability to fully see.
Only in brief reflection do I come to mind.
We can, however, see our eyelids when looking in a mirror, or, to go deeper with it- when we close our eyes to reflect on past choices or anything, we acknowledge that our eyelids are in front of our eyes. In other words, they come to mind.
So again, you are
Eyelids