Removing them will work, but the polin may be floating around in the area, I would wipe the floor down, because the polin will fall with gravity, but a fan may circulate it to other plants.
Taking off the flowers, may help, but do you see any balls, or balls that look like they exploded?
Adieu