Are you perhaps
Sunday / sundae?
I'm here for your confession,
At a church, on a Sunday.
And I'm found in a progression,
Days of the week.
I might be found in a creamery,
A sundae is a kind of ice cream.
I'm also named for a deity,
Many religions have, one way or another, worshipped the sun.
You seldom work when I'm around,
Sunday is a day of rest in many parts of the world.
Which has me feeling kinda blue,
Not sure whether there's any specific clue here. (But the so-called "blue laws" were all about forbidding various activities on Sundays.)
So you ate me by the ocean side,
Sundae again (though I'm a bit unconvinced here; usually the ice creams you'd eat by the seaside are simple cones, and sundaes are more often served indoors in some kind of dish.)
While the salty air gently blew.
If there's a specific clue here, I'm not seeing it.
"When you ate me I was cold": sundae again.
"I shouldn't be interpreted as being salty": is this just forestalling guesses at the wrong kind of food?
"When you're done with play, I'm yesterday.": on a Monday, when the working week resumes, Sunday is yesterday.