if it's the only plant you have growing, then there is no harm in letting it grow until you're more than sure it's male...

after all, it could be female. I could be wrong.