Yeah man, you just have to be firm and show them that you're in charge of yourself not them. That goes for anything in life. You don't have to be an asshole, that might start a fight but just say, "No, i've got shit to do" or something and then walk away with confidence.