You may have a light leak, or heat stress.
A couple nanners aren't going to completely ruin your crop. Go through carefully and check daily, picking them off with tweezers, if you suspect you are having herm problems. It's a good practice to look for herms even if you think everything is fine.
This will sound anaethema to some growers, but my opinion on seeds in your bud is: Oh well, it has a couple seeds in it. Boo freaking hoo.
Do I pick off any herms? Yup! But a seed here and there isn't something that I've ever lost sleep over, and it really doesn't make your yield go down or anything like that just having the occasional one scattered around. Hell, the guy upon whom I offload my excess gets all SORTS of excited when he finds a seed! It's cute, this big burly thug-lookin' dude getting all giddy over new genetics to play with.
You aren't screwed.