There are a couple types of brown- the dry, flaky, light-colored brown which is just bark (yup, cannabis grows bark when it is old enough!) and a darker, wetter-looking, more sinister brown tht comes from fungus.
If it's flaky, don't fret.