No problem with properly-draining clay pots per se, but your life will be easier if you use plastic because when it is time to transplant the whole root ball just slips out of plastic, where with clay, you sometimes have to break the pot.