Research indicates that light use actually helps mental function, but heavy use impairs it.

"light use" is less than five joints' worth per week.

I can't speak to the sperm count thing, but "burnout" is a real phenomenon. It differs from person to person but you can enlist a spouse or best friend to keep you in line if you feel like you can't gauge it yourself.