In my opinion, it doesn't matter what size container you use... One thing to keep in mind is that its tricky to transplant when using all perlite or perlite/vermiculite mix...
It can be done, ya just have to be careful. Starting them in a small, tall container like a soft drink cup, letting them get a nice root system, and then transplanting into the container it will finish in would be what I suggest.