What to do is, use tweezers to carefully remove the bananas, then hope not too much if any of the pollen falls. If you can get to the male bits early enough they wont even have formed pollen yet.

The second thing to do is watch for seed development, squeeze the buds a bit to see if they are forming in there, if there werent many bananas you wont get all that many seeds if u let them go the whole flower. Its a bitch to watch for male flowers everyday but to ensure mainy sensi its worth the effort.

PS Have u found the reason for your hermies?