Stop smoking for a week, THEN go have your breathing checked. You may have problems unrelated to cannabis, having more to do with your respiration in general.
Respiratory problems can cause a 'panic' response, even if you don't notice a breathing problem yet.