Its just a tolerance thing. Pot is not a sedative like many think its actually somewhat of a psychedelic stimulant. If you smoke potent herb with no tolerance, a panic, or anxiety attack is very normal. Hallucinations, inability to sleep are also symptons that can happen with low tolerance, potent herb.
I have friends that don't smoke ever and get bad attacks when they do. I also had to quit for a year and when i started i had them bad. I just kept toking and they went away. I do still get them here and there although rarely and I smoke everyday. Sometimes it sneaks up on me though
If your having anxiety and pot makes it worse its just pulling that issue to the forefront of your brain and making you deal with it imo. Keep toking and it will work itself out or try a less potent strain.