I never remove leaves. But I DO remove entire stems righ tbefore flower. Anything that is heavily shaded and therefore weak, soft, and spindly, or thin and scruffy with stunty growth, gets cut off. You are correct to thin the CENTER of the plant; this is a traditional goal of pruning in horticulture. What it does is allow the plant more air flow to the center, reducing chances of rot or other disease, and it also forces the plant to concentrate on the prime, high-value main colas which are already vigorous.
For an extreme example, google the term 'pistilwhipt technique'. I don't go THAT far, but pretty close!