It seems that when my tolerance is lower I feel more prone to anxiety... You might just have anxiety issues, I have them myself but I have made it past the feeling.. My anxiety use to raise when I smoked too much [dosen't happen anymore].. Now usually if I only take a couple hits it will just feel like my anxiety is raised.. It's a weird balance of things, at least for me.. Everyone is different. I hope you figure it out man!