Once you notice frost in the morning then it's getting getting too cold. If they aren't done yet I wouldn't pick them, if you get some frost and it kills them then you have no choice.