This website has some info on them. Unless your plant is very young like seedling status I think it will be ok with them there. You can just pick them up as you see them, they're harmless. Just throw them outside. And I've never seen them really burrow. I only see them like on moist soil under say a log. Sift a little through the topsoil and you should get any that you couldn't see anyways.

What you're suggestion, I'm not sure if it wouldbe good for the plant, but it would probably get rid of any other bugs that might be in the soil. Wait for some advice from the experienced members though.

EDIT: Oh, and that sig of yours, lol, 5th of everclear? That'll get you fucked up lol.