Presuming they cant fly..

I would rub vaseline around the bottom of your pots if they crawl up from the bottom or just at the bottom of your plants stem to stop them climbing your plant. If they can fly, i wouldnt know what to do, maybe organic pesticide !!

good luck
Charbud