Yeah usually once I see some vapors swirling, I'll leave it for about 5 more seconds then take the hit immediately. As for the harshness, this may depend on the bud you're smoking, but generally people aren't ready for a new sensation when inhaling vapor because it actually may be hotter than the smoke you inhale. It is also 99 percent THC in the vapor, the purity of the vapor may also have an impact on how it feels upon inhaling. Leaving the bud in one spot on the bulb may burn it and you may end up inhaling more smoke than vapor. Sometimes the bulb may get hot enough to continuously vaporize THC without a flame under the bulb. This is generally the most efficient way to create the vapor but it may sometimes be hard to coordinate and find the best time to take the flame off.