you can, but it's not the best way to go about it. If they're already in there, leave them. Are you going to veg for a while, or go straight to flower? I recommend vegging for a few weeks to a month in those pots. The issue with putting plants in too large containers is that even with good drainage, the medium stays wet too long because the root mass hasn't grown enough to be able to take up all that water fast enough. When the medium stays wet for too long, it can promote mold and mildew. Cannabis like a wet/dry cycle and that's hard to control when the pot is too large for the plant.
It's also more difficult to determine when to water since the top few inches of soil can be completely dry, but the bottom can still be very wet. If you go straight to flower, this amplifies the issue since the plant doesn't spend as much energy growing roots at this point.

I've done that once out of necessity and I'm never doing it again.