Where to begin!
First you get the problems all farmers face:
Insects
Deer
Bunnies
Woodchucks
Other puntable furry things
Drought
Poor soil
Fungus
Then you get all sorts of other fun stuff only cannagrowers have to worry about
Hunters
Cops
Cops in helicopters
Do-gooders
Birdwatchers
Pot scavengers (my favorite, and one of the few times in life I would actually consider beating someone senseless with their own severed arm)

To lessen the risks,
-Scout well and that includes digging test holes and taking soil samples- test them for pH and also check what the soil is made of.
-Keep your mouth shut
-Make the site hard to find or undesirable- pricker bushes are your friend
-Check that access is not going to be suspicious; if you have to park a vehicle, it should not seem out of place. A recreation area where people walk their dogs is a good place to park. Take your dog or borrow someone's. Make it look like you have a reason to be there other than being a sketchy potheaded lil criminal with a shovel and a dream!