Are you going out there to feed your babies at all? If you are just give them the wipedown with a damp and dirty rag. If not... well I would say that it is probably no big deal, but you never know for sure.
What part of the US are you in? I am in the NW and there arent too many cops who would care enough to bust someone for a plant or two. Good luck to you.