You most likely have a pest known as corn borers.
The only thing I know to do for them is mostly only preventative, and won't help once they are already in the weed.
Next time you grow in that spot, sprinkle some boric acid on the ground around your plant. Be careful not to get any ON your plant, as it can cause stress, and hermaphrodites.
The good news is, they won't much effect the quality of the buds they don't eat.