Note: this answers the question before the question was changed from "left alone ... with" to just "left ... with".

Definitions: 'trip across' means going from the original side to the destination side; 'woman left alone with a man' means those individuals are the only two people in that place.

Send all the women across first. This requires $n-1$ trips, leaving the boat at the destination. One woman (call her W) returns.

Send all the men across. This requires $n-1$ trips, leaving the boat at the destination. W's husband returns and both go across (1 trip).

>Total $2n-1$ trips across to the other side. Each trip across is matched with a trip back except for the last trip across, so we have $4n-3$ river crossings.

In the first phase, every woman is in the company of either only women or with her husband. In the second phase, W remains with her husband or is alone, and all other women are with at least the number of people they were with in the first phase.

This is minimal since each pair of river crossings can deposit only 1 person at the destination if someone must return with the boat. After $2n-2$ pairs of river crossings (and hence $2n-2$ trips across), only 2 people remain on the original side. They take the final trip across to total $2n-1$ trips across.