No problem, as long as scrubbed air is re-circed w/ in the hut, and not exhausted.

If you can smell from outside the room, suggests air exchange even when vent sys off- you're losing CO2. You might want to think about a back-draft damper on the exhaust, if yer fan will handle it.
Also make sure little flap-covers on the intakes to the hut are closing all the way.

Often a good idea to run duct from internal scrubber to top of space if filter low, bottom if filter high- you can use it as a de-strat as well.