That is exactly what is happening. By reducing the size of the exit point you are creating backpressure on the fan. Think about it in actuality you are bottlenecking the exhaust and will shorten the life of the fan itself. It also depends on how much space there is in your starting size and your end ruduced size. Naturally if you took a 10" fan and reduced it to a 4" in a matter of two feet, the fan is going to be struggling like a bitch to get the air out..somewhere..LOL. The fan itself also is working harder but no air is being passed through it to cool the motor. The fan relies on incoming air to cool it off as well.
Hope this helps.
Tryna