Not all chemicals disperse when you let the water sit. The maximum amount of dissolved gases that can remain in solution is related to the temperature of that solution, so cold water can hold more than warm water- as the water warms to room temperature, some will come out. Many chemicals will remain in solution, especially minerals like lime and sulfur, which do not exist as gas at room temperature. Only compounds which exist as gas at RT will leave the solution.