Coffee filter material should work.
Baking soda won't bring your pH high enough, nor vinegar, low enoguh.
Your rain systems should be relatively clean if they don't pass through a highly industrialized area. Our rain here in New England comes via the Rust Belt, including Sudbury ONT... lol... it's a bit on the acid side...
Roof runoff should not be bad unless you have a rusty tine roof. REmember, the rain is constantly rinsing your roof down. Wait until the rain has been falling for half an hour and then open up the inlet to your rain barrels. That will give dust and contaminants time to wash mostly away before collection.

You don't lose water with an RO system. They operate under different conditions than osmosis in plant or animal tissue, and are both effective and efficient. IF you can afford an RO unit, you are GOLDEN, and if you run it starting with rain water, the filters should last a lot longer between replacements since the EC of rain water is already quite low.

If you choose rain water, first test the EC, then bring it to 300ppm with your CalMAg Plus, THEN add the ferts, THEN check and adjust the pH.