males will not produce pollen until their sacs are full which happens early in the females' flowering time and continues for 2-3 weeks (assuming they're the same strain of course). growing side x side during veg will not hurt anything other than limit the space your f's have but be aware males need to be gone BEFORE they produce pollen but they give plenty of sign before that happens.

4 males out of 80 is unheard of so expect to find a lot more...