I agree with folks who said you should go ahead and harvest some if they become ready and no one has done so.

Lots of us weed-friendly folks purposely scatter some of our extra seeds in semi-secluded places in the hopes that they will grow and someone will happen upon them and enjoy!