You are on the right track with the fans and noise reduction. A higher CFM fan dialed down will be quieter than lower CFM fan running at top speed. Also, noise can be greatly reduced simply by up sizing the ducts. So if it's a 6" duct, taper it up to 10" south of the fan and you will see considerable reduction in noise. I would do that before doing any experiments. That is essentially what a DIY duct muffler does anyway. See link in my signature for good info on fan sizing.