After reading a whole bunch of threads on OG about those fans I've come to the conclusion that they make a considerable amount of noise due to three following factors.

Vibrations
Echoes
Air Movement

Most the people have suggested the following to take care of these.

Vibrations: Most have the fan simply sit on the floor of the chamber. To have it hanging from something (strings, cords, etc.) would eliminate the vibration factor due to the strings absorbing all the vibrations and not transfering them to what they are connected to (typically the ceiling of the chamber).

Echoes: This noise is basically the noise that the machine itself makes regardless of if you're holding it in your hand or if it's drilled to the floor. Knowing people in my family who are into sound equipment and studios I know theres a material used in Car Audio that is called Dynomat that you can line the walls of the chamber containing the exhaust fan in. You can also use other things ranging from a cork type material to styrophoam to egg cartons (the cardboard kind).

Air Movement: This noise is caused by the airflow itself and doesn't really create too much noise in itself. But if the exhaust fan itself is connected directly to the ouside of such a box then you would probably hear it (and you also wouldn't be able to have it hanging from strings if it's touching the outside). This can be fixed by adding ducts or something to connect to the fan to then bridge the connection to the outside.

Also another thing that people pointed out was that if you get a bigger fan it would make less noise if pushing the same amount of air as a much smaller fan. This is because of the bigger fan needing much less RPM to match the smaller fan hence reducing the noise level.

The reason I'm pushing the silence factor is mostly due to completing the perfect stealth box. Of course people would notice a ginourmus bix sitting in my closet, but that's not what I'm talking about. What I'm talking about is that if done properly, when you approach this box, you will not smell anything (maybe fresh air from carbon scrubber), you will not see anything (no light leacking), and you will not hear anything. This in turn would allow for growing without ever needing to take up too much space or be annoyed by certain parts of the grow.

Thanks for any ideas/comments.