I think its a conditioned response, as can be seen with twin studies. The reason we dont like asymmetrical animals is because of what I think is called the good gene theory, which means that we choose our mates based on who we think has the best genetics.