The sailor did not force him into the situation. If anything, he put himself "under duress". The sailor has no obligations regarding the drowning man. Coercion would be initiating force against someone. For example, kidnapping someone, stealing from someone, hitting someone.
You haven't given any reason why it should be relevant who put the drowning person into a desperate situation. There are circumstances (like the duress of a drowning person) in which consent can't be given, and in general it doesn't matter who, if anyone, originally brought about those circumstances. Why should coercion require the initiation
of force against someone, rather than just the exploitation
of force--regardless of origin--which is currently acting on someone? The causal origin of the force doesn't change the force's ability to make people agree to things out of desperation which they don't really want to agree to. That ability is the underlying reason why intentional threats remove the capacity for meaningful consent, and the principle applies irrespective of the nature or source of the force.
Let's try a different thought experiment to illustrate this point. You're walking by yourself along a poorly-lit street at night, and someone mugs you at gunpoint. "Give me your wallet, or I'll kill you," he says. It's agreed that in this situation, if you were to give him your wallet, your doing so would be coerced rather than consensual. The fact that you technically have two 'options'--compliance or death--doesn't change the fact that you can't consent
to trade your wallet for not dying. While you're considering your 'options', a passerby comes along and sees what's happening. He says to you, "Ah, I see you're in a tricky situation here. I'll tell you what: if you give me your wallet, I'll beat this guy up for you, and you can go home safe."
According to your position, giving the wallet to the mugger can't be done consensually, but giving the wallet to the passerby can, since the mugger initiated the force--put you under duress--whereas the passerby didn't; rather, the passerby is merely taking advantage of the duress you were already under. But as far as you, the victim, are concerned, your options, motivations, abilities, and desperation are identical regardless of whose offer you take up. Both the mugger and the passerby are presenting you with the same dilemma: giving up your wallet or dying. If deciding to give your wallet to the mugger can't be considered 'voluntary' in the relevant sense, then neither can deciding to give your wallet to the passerby. From your perspective, the situation is bleak either way, and your alternatives are the same regardless of who--the mugger or the passerby--is responsible for your plight.
Furthermore, you can't even be certain
who's responsible for it. Suppose the passerby is actually a mad scientist and criminal mastermind who implanted a mind-control device in the brain of someone who's actually a nice guy, turning him into a mugger. The passerby orchestrates the situation so that you believe the mugger to be the one who brought you under duress, when really the whole episode was planned out by the passerby. Then, by your argument, giving the wallet to either
of them would be coerced rather than consensual. Or suppose it's the other way around: the mugger is the mad scientist mastermind with control over the passerby's actions. Here, similarly, giving the wallet to either of them would be coerced, by your argument. Yet for you
all three of these situations are utterly indistinguishable
On your conception, consent is something that we can't even know we're giving or not giving. Time to get a better conception.
If you charge a drowning man to board your ship, you're being a jerk.
You still aren't giving an explanation. Why is that a jerk move? On your view, the arrangement is perfectly consensual, and all parties involved got what they wanted out of the deal, so what's the problem?
You know the answer, but you also know that you can't admit it without conceding the point. The answer is that people in desperate situations--no matter how they got there--are, for just that reason, not in the position to give meaningful consent.
One more thought experiment. Compare this scenario with one in which the person in the water is a competent swimmer, is in absolutely no danger of drowning, and happens to like checking out people's boats. The sailor comes along in a nice boat, and the swimmer says, "Hey, cool boat! Do you mind if I come aboard and look around?" The sailor turns out to be a private and possessive type, and he doesn't want the swimmer on his boat. So he says brusquely, "Sure! If you agree to give me your life's savings [...etc...], then you can come aboard." On your view, where the validity of the arrangement apparently isn't affected by the fact that one of the parties is drowning, this proposition should be equivalent to the one the sailor gives to the drowning person. So I guess the sailor is just as big a 'jerk' in this scenario too?
No--the difference you refuse to acknowledge is that the drowning person is being coerced by an opportunist, whereas the competent swimmer isn't.