I think since your in Maine(if your in Maine), You really have little to worry about. I have many friends living in Maine. I spoke to a few and they said that Maine is next to Cali. as far as weed. they said that for 2-4 Plants outdoors, it might not even be worth there time. they said that there only looking for "more than personal". they also said that worst case they could think of would be them coming to tear them up.
I never grew in Maine so I never assessed the risks but my peeps do grow and they didn't sound as if this was really an issue. They were saying that they may have noticed one and been searching for the cashcropper stash.
Good luck with this. I'll be hoping for ya.