I believe that you are in the
House of Dionysus in Paphos, Cyprus?
To the North, take a horse to find something delightful
A famed horse, the Byerley Turk, was one of three horses to be forefathers to modern thoroughbred racing. The something delightful is Turkish Delight seems to indicate Turkey is to the North. Per @jafe, the horse refers to the Trojan Horse, and the fabled city of Troy is believed to be in Anatolia. Thanks, @jafe!
To the West, a game piece that was tight with Paul
A game piece in chess is the Bishop, @Gareth McCaughan found that there's a pun in the word "tight": the Titus to whom one of the allegedly-Pauline letters in the New Testament is addressed was the Bishop of Crete. Thanks, @Gareth! Crete is to the West.
To the South, he can't smell, although he wasn't spiteful
This could be the Sphinx (since he's missing his nose!) in Egypt, which is to the South.
To the East, you might see lavadeserticus crawl
This is the Syrian house gecko, endemic to Syria which must be to the East.
Drink and be merry while you admire the floor
At the entrance of the house is a floor made of shells depicting Scylla. Dionysus is also a God of drinking and being merry in the Greek pantheon.