Everyone cares what other people think of them. If you truly don't, it's because you've given up on life. I guarantee you, that you at least try to wear decent clothes. You probably shave every once in a while, and maybe even brush your hair.
You don't do that because it makes you feel good about yourself. You do that because you want to give the right impression to those around you.
Similarly, everyone censors their own actions around other people too. You're probably a nasty bastard when you're by yourself. You probably don't shower first thing in the morning, if you're just going to be hanging out alone in your apartment. You probably don't wash your hands with soap and water every time you go piss. You probably talk to yourself, and sing in your car on the way home. But if you're in public, you're going to act differently. It's human nature.
And that goes, too, for what you wear. When you go out to a club, do you wear the same clothes you wear when you're lounging around your house watching TV? Of course not. You try to give off a good impression. Why shouldn't that apply, too, towards what you drink? You might love a cosmopolitan, but chances are, you're not going to order it for yourself, when you're trying to hit on that chick down the bar.
I can't fucking stand when people try to act like they're "better" than that. When they think they don't have to try to impress people. Where they just "act like themselves". It's bullshit, and everyone knows it. You want to claim you're a man? Then admit that sometimes you do what it takes to make a good impression. Lying about something so plainly obvious is pathetic.
PS. Bodybuilder's are typically not "manly" in the usual sense. When trying to defend your manliness, I'd probably leave that detail out.