You are:
baking a cake! Specifically, a pound cake, as you are creating a 'mix' (as per the title) of equal parts BUTTER, SUGAR, EGG and FLOUR.
My prefix is something located behind,
My suffix a place I go with worry on my mind.
BUTT ('behind') + ER (Emergency Room - 'a place I go with worry...')
My prefix is wind, albeit reversed and then headless,
My suffix noble, and helps to be soundless.
SUG (i.e. GUS(-t) reversed) + AR (Argon = Ar, a 'noble' gas, used in double-glazing for better heat insulation though often popularly thought to be used for soundproofing)
My prefix rises quickly, when adding numbers to its core,
My suffix a goodbye, used by digital competitors.
As solved by Rand al'Thor: E (the mathematical constant, e, representing exponential growth) + GG (gamer speak for goodbye)
My prefix is known for its swamps and men,
My suffix was mine, but I hadn't bonded then.
FL (the abbreviation for Florida, known for the swamps of the Everglades and 'Florida Man' headline memes) + OUR (after partnering up, what was 'mine' becomes 'ours')
All of this explains the reference to 'flavor' in the hint also :)