Not necessarily.

In moderation, man-made drugs such as crack, and meth are having much less daunting side effects in the short-term than naturals such as shrooms and cacti.

In the long term, obviously we know what worse, there no need to discuss.

In essence you really gotta judge the drug uniquely.