I definitely agree with your sentiment but sometimes people do make lighthearted comments that get misunderstood and/or written off as stupidity.

You'll drive yourself nuts if you let that sort of crap truly bother you. There are a lot of idiots and you can't do anything about it, but you can choose to ignore them and accept that not everyone thinks the same way you do, and just that alone doesn't always make them an idiot.