No, not fertilizer. It's a mixture they add into the soil to make the soil lighter. It makes the water easier to soak into, and the soil won't be as hard, which can make roots weak as they can't push through the dense soil, so they use perlite or vermiculite to lighten the soil up.

It's propbably not great to smoke, but it isn't a fert.