Don't conform to what your friends expect from you. You should be the person you want to be, doing the things you want, as long as they don't hurt anyone else, let alone yourself. I think it's great you want to quit for a while and save money and tolerance, but if people are saying things about you, confront the issue with your words.

There is more to life than these stupid people.