Is your left testicle sensitive, to some degree or another, without the weed? Does it sometimes hurt by itself? You could have torsion in it, and the vascodilation causing an increased bloodflow to it intensifies the pain, or increases the inflammation just enough to hit the right nerves.

Try gently icing you testicle from behind, the side, and on top. You may want to try this before smoking to see if it alleviates the effects, and thus come closer to an accurate diagnosis.