I would just tend to it, picking off the balls. Hermie's usually don't produce a whole bunch of male flowers. I've had a couple in the past and after picking the balls off they turned out fine. Once I missed some of those buggers but still only had 20 or30 seeds in the whole garden. Nothing wrong with getting rid of hermies, it's really a personal choice.