Sorry, but there are logical inconsistencies in the positions resulting from the most basic flaw in Libertariansim. Libertarianism is based on the concept that people should be allowed to lead their lives without interference and that they will then do the right thing.
This ignores the reality that few people are actually live on the basis of enlightened self-interest (called rational anarchy in "The Moon is a Harsh Mistress, the Robert Heinlein book sometimes credited as the basis for Libertarianism). Even the best people are prone to letting greed, fear and other flaws influence their actions. That's what makes government necessary as well as problematic.
Consider the example given, that the sexual orientation shouldn't be an issue, that people should simply be allowed to live their lives. Fine, except that many people are unwilling to let things work that way. Too many people want to impose their views on others, by force if necessary. It hasn't been long since race and gender affected access to basic rights, and we really haven't resolved that problem. Such rights are also frequently denied on the basis of gender orientation. That's why the government has to interfere.
It's all well and good to say that people should be free of government interference, but since too many people never learned to work and play well with others, interference becomes necessary. Limiting that interference to the necessary level is important, but cutting it below that level is not in our enlightened self-interest. (And reading "The Moon is a Harsh Mistress" actually explains why Libertarianism is a fine individual philosophy but absurd and doomed as an applied social philosophy.)