it depends how numerous they are. but it's not the seeds you want to be plucking anyway, it's the male flowers; those little, yellow banana shaped things, usually hanging off the BOTTOM of the buds (look closely underneath the buds). it's those that will spread their pollen and infest the rest of the plant. if they aren't too numerous (i would say less than 15 bananas) then i recommend plucking them, and doing it very carefully so as not to agitate them and thereby spread the pollen. or just leave them on, your choice. the plant's already experienced some level of stress, and there's nothing you can do at this stage to take that back. all you can do is ensure that it lives the rest of its life stress free. i'm sure if you correct some of the mistakes you made this round on your next grow, you'll get much better results. as you gain more experience, you should see less and less seeds in your grow, even if you're working with bagseed that is likely to hermi every time