imo ...more info is needed to make an educated guess.

But if I were guessing, and I've dealt w/them, I'ld say the gnats are doing you in.

Neem oil will kill the adults, but the adults do not eat. The larve in the soil eat roots by boreing into the roots. neem oil is no good for this. also! nothing will kill eggs, and they can survive drought and flood. screens on windows will not keep them out, not small enough.

get some gnatrol or mosquitoe plunks(much cheaper) I can tell you how to use later. mtc