if you have a lot of plants, and dont want to pay attention to this one all the time, you could dispose of it. you could also carefully pick off all of the balls when you see them growing in with some tweezers. A good idea is to spray your plant with water before this so none of the pollen can get airborn if you break the balls.

Remember when picking them off, the balls that grow on a stem are the ones you want to pick off. if it doesnt have a stem and has 2 hairs growing out of it in a V shape, its a female part so dont pick it off. Do this before they break open, otherwise you will get a bunch of seeds, which will hermie also.