Are you sure he is 100% comfortable with you being around him while he is high?

You say his girlfriends in the past gave him a hard time about it? If a smoker gets bad vibes from non high people when he/she is high, sometimes that can make them uncomfortable around non high people. There might be leftovers in his subconscious that send him the message "don't get high in front of the GF!".

Even though he knows you are ok with it, the subconscious is a powerfull thing. He might just be feeling awkward because he knows he is high and you are not. He might be thinking things like "she is pist or disappointed in me". Even if he doesn't admit it, these feelings could be happening.

I know when I am high and around people who had a problem with weed in the past, (like bosses), and I am around new people who don't mind (like my new boss), I still get that leftoever feeling of "Are they pissed at me?"

It is not always rational, but when your high, rationality takes a back seat to emotions. Sometimes this creates paranoia. I blame the Governments of the countries that made marijuana illegal. It created a whole stigma.