My setup is somewhat different than yours so my methods may not apply. I use a Hydro Hut so essentially I have a grow room within a room and I am able to keep the cool tube sealed off completely from the grow area - air comes into the tube from outside the hut, through the tube and is expelled outside the hut though a window - that way the only odor I have to control is from within the grow area which is where my carbon scrubber is attached to the squirrel cage to exhaust, circulate, & scrub the grow room air. Intake is passive through vent flaps.

Your closet area has different requirements and the fact that you don't want to do any hole cutting has its on limitations. Is the closet completely sealed from the other areas? You may end up using your original idea and it might be perfect for your setup! Keep us up to date so we can all benefit from your experience.