You are:
The Buddha statue at Lahaina Jodo Mission, a Buddhist temple in Lahaina, HI (on the island of Maui)
To the North I see headstones, but to my surprise - no grass, no dirt, and no clay.
Just to the north is the Puupiha Cemetery, which according to a picture of Google Maps has headstones in sand (not grass, dirt, or clay).
To the South is the smallest you probably can't visit, then nothing for a very long way.
Going down south a little further is Kahoolawe, the smallest main volcanic island in Hawaii, which is closed off to tourism. And then just ocean until Antarctica.
To the West is a garden fit for the gods with very few things that decay.
Refers to the Garden of the Gods, which is on the island of Lanai, just west of Maui
To the East is a needle that cannot be threaded and needs very few letters to say.
Refers to the Iao Needle, a mountain east of Lahaina
They say outside where I came from, there are none larger.
The statue was brought in from Japan, and happens to be the largest Buddha statue outside of Japan.
And now to address the hints, many of which refer to the directions rather than the place itself:
Hint 1
If you go far from any location in Hawaii, you'll just see ocean.
Hint 2
"Iao" has very few letters and no consonants.
Hint 3
Kahoolawe is the smallest of the eight main volcanic islands of Hawaii.
Hint 4
"Zen" in "MetaZen" refers to Buddhism, and the location is a Buddhist temple.
Hint 5
Just to the north of the Lahaina Jodo Mission is the Puupiha Cemetery.