I'd trim off the low bud sites ONLY if I were growing in a hydro/RW/automatic drip system, and then, only after they have put on some height, and only to help with air circulation over the media, and for access to the drippers.

A clone that was taken as only a single growing tip will continue to grow as a single-lead plant anyway. In gallon pots, the spacing between the plants should still allow for decent light penetration. Personally in that case I would leave them unless it is a strain that you already know puts on REALLY nasty, scraggly low shit.