Stems contain a minute amount of THC and any THC found in the stem is usually just the sticky trichomes from the bud that ended up on the stem.

Not to mention the taste of the brownies would most likely be pretty harsh.

Just things I have read in the past. But if you are going to do it, try at least throwing in some shake so that the attempt doesn't end up a complete failure.