The whole controversy is utter bullshit.
How do you all feel towards chemicals and natural psychedelics?
First of all, all psychedelics are chemicals. The ones that are not 'natural' are better reffered to as 'synthetics'. Then there are plenty of 'natural' substances that are dangerous to humans. Likewise there are plenty of 'synthetic' substances that are virtually non-toxic or that have a low toxicity.
Most synthetics are also derived from natural compounds. Outstanding examples of this practice can be found in TiHKaL and PiHKaL. Alexander Shulgin has synthesized and bioassayed hundreds of tryptamines and phenethylamines over the years and he seems to be doing quite well.

It should also be noted that plants and organisms are essentially synthesizing substances themselves. They didn't come from no where, the organism is capable of building certain compounds. Mushrooms for example are capable of adding a 4-hydroxyl group to any tryptamine in its substrate. Usually it does this with DMT to produce 4-ho-DMT aka psilocin, but you could throw in a tryptamine not known in nature like DET and the shrooms would 4-hydroxylate it to 4-ho-DET.