You are

A halogen lamp!

I am fragile

Indeed, you're made of glass

and in many colors do I come,

There are coloured halogen lamps

You can touch my cap

You can handle a halogen lamp by its cup-shaped reflector

But if you touch my stem

But if you grab it by its cylindrical bulb...

Unfriendly air will you meet.

Once lit the grease from your fingers will cause it to explode and release the halogen gas inside.
"Unfriendly" might be a play on words with Bromine as in "bro" (a friend) and "mine" (the explosive device).

You are

A halogen lamp!

I am fragile

Indeed, you're made of glass

and in many colors do I come,

There are coloured halogen lamps

You can touch my cap

You can handle a halogen lamp by its cup-shaped reflector

But if you touch my stem

But if you grab it by its cylindrical bulb...

Unfriendly air will you meet.

Once lit the grease from your fingers will cause it to explode and release the halogen gas inside.
"Unfriendly" might be a play on words with Bromine as in "bro" (a friend) and "mine" (the explosive device).
Much more likely is that "unfriendly" refers to the inability of Noble gasses to form bonds. However, noble gasses are not halogens, and noble gas lamps (such as neon tubes) do not have any special handling AFAIK.

You are

A halogen lamp!

I am fragile

Indeed, you're made of glass

and in many colors do I come,

There are coloured halogen lamps

You can touch my cap

You can handle a halogen lamp by its cup-shaped reflector

But if you touch my stem

But if you grab it by its cylindrical bulb...

Unfriendly air will you meet.

Once lit the grease from your fingers will cause it to explode and release the halogen gas inside.

You are

A halogen lamp!

I am fragile

Indeed, you're made of glass

and in many colors do I come,

There are coloured halogen lamps

You can touch my cap

You can handle a halogen lamp by its cup-shaped reflector

But if you touch my stem

But if you grab it by its cylindrical bulb...

Unfriendly air will you meet.

Once lit the grease from your fingers will cause it to explode and release the halogen gas inside.
"Unfriendly" might be a play on words with Bromine as in "bro" (a friend) and "mine" (the explosive device).