i wouldn't keep any of the seeds from this crop...they all have the potential to be hermaphrodites with the slightest amount of stress. it's very time consuming to try to "prune" off the male flower and you will just end up with more seeds that are likely to switch on you at the end of next year's harvest.
if you have removed the plant with male flowers then you won't have any furthur seeds develop, but there may have been some pollenation prior to harvesting the hermie and the seeds are so small that you may not be able to see them all yet.