Im pretty sure those are small caterpillars, and even if theyre not i had to deal with em too. They rest underneath leaves in a half chrysallis (i know i didnt spell that right) made out of some resinous webbing. Well anyway theyre hard to get off with soap sprays and such so i just removed them by hand and as far as i can tell theyre not too great at reproducing. You should be in the clear But google some anti-caterpillar/green worm stuff anyways just to be on the safe side