Replies to questions by forum members and ex-forum members
You don't have any a priori "legitimate" right to live, outside of an agreement inside society representing the exchange of that right (mutual recognition) between the members of society.
Legitimacy doesn't simply float in the air. It is a question of mutual recognition of a right inside the system of exchanges we call society.
And for the exchange with the whole, the implicit moral exchange is that an individual has no right to do what harms the whole.
Can the whole always a priori recognize or not if something is harmful to it, is a different question. This thing is done a posteriori, by analyzing the consequences for the whole, but in the context of a preexisting moral frame of exchanges between the individuals and the whole.
And it is the whole that decides (by whatever social mechanism it establishes), not just individuals directly affected, like you claim.
Btw. the issue of practical decisions has to be partially discriminated from essential moral issues.
For example, even if it is established that a behavior is harmful for the whole, a society might not have the resources to weed it out efficiently, and then must, at least temporarily, weight if its forceful efforts to do so are productive or counterproductive in the sense that they can do even more harm, and adopt other ways to progressively curb such behavior.
But such activity by the whole is also the consequence of an agreement or lack of agreement about giving the right to the whole to decide if some behaviors are acceptable for the whole or not. This is a practical issue concerning the question of members of a society giving or not giving that right to society for everything/something, or not, and can they collectively live with the consequences of that decision.
(If they decide wrong, that can even lead to the destruction of society.)
All those things are implicit and often poorly defined, but a correct theory of morality should precisely make it all explicit, or more exactly, give a correct frame for the moral explicitness in any kind of situation, so that things can be weighted correctly inside a frame defining the relations of exchange involved.
The reason why it is important to understand morality as exchange based, is precisely in areas concerning the exchange with the whole, where some ideologies claim that some rights exist by themselves, and that the "natural" rights of some people are infringed if one does not allow them to do something.
Without the correct frame of understanding, it is sometimes hard to find the right fundamental arguments why it is not so.
A correct theory of morality allows moving from ideological arbitrariness and subjectivism, to an objective analysis of moral situations.
It doesn't mean that in some cases even such analysis cannot result in a dilemma, it can, but there are other cases that are far more clear cut morally, but are muddied by purely ideological false beliefs and choices.