I think that's probably the best bet.
But yeah, think about it. The fan itself takes up physical space in the hole. If it is not moving so fast that it can keep up with the air movement of FOUR similarly-sized fans, it is blocking air movement.
It can be difficult to tweak an active intake correctly, which is why few growers bother with them. Even commercially-produced grow boxes have passive intake.
I like the use of a prefab tool closet for a grow chamber. I haven't seen many posted- I think xcrispi has one for his moms n clones- but they are neat.
When/if you outgrow that box, it will be IDEAL for keeping a big mother in the bottom half and clones in the top, to stock up a room up to about 6x6 feet comfortably!