I think you are a(n):
incandescent bulb (common light bulb)
Clues + explanations:
I am fragile
Light bulbs are fragile.
and in many colors do I come,
Light bulbs can have many different colors!
You can touch my cap
You can touch a light bulb's "cap" (which is the glass housing around the bulb).
But if you touch my stem
The "stem" of the bulb is inside it. This is key for the next clue!
Unfriendly air will you meet.
Thanks to @Quentin, we know that this clue has to do with the inability of Noble Gases to form bonds. Argon (a Noble Gas) is a common gas found inside light bulbs, so if you're touching the stem, you're gonna meet a Noble Gas. Probably not for long, though, if the cap is off. :)
HINT 1
I am a non-living thing
Light bulbs are not alive.
HINT 2
I am an everyday object
You find them everywhere!