I dont really believe there is a pat answer, as we all have differnt views and wants and desires.
My lady and I met in a swinging atmosphere. She was single for many years and not looking for a relationship. I had been single for 4-5 years and not looking also.
At one time in my life I could have not even considered letting someone I care for have sex with another person. With my lady tho, I simply adore seeing her pleasured and pleaseing, whomever she is into, a man or a woman. It's not a thing we do every day nor every month, sometimes a year or more goes by then, something will present it's self, and if we're in the mood, cool! If not, it makes for some hot talking that night....and killer good session with her.
We are older tho, our minds and bodies are not clouded up with raging hormones like the young, and I think that is most of the jelously, untrusting, and worry comes from.