That would piss me off too. You have absolutely every right to your own medical records with the only exception being phychology notes, which doesn't apply to you, just sayin'.

Definetly try to keep it civil and remind the office that he must give you your records, according to law.

There's already case precedent protecting physicians from any retaliation for recommending mmj, but just handing over your records will certainly not put him on any list.

The guy is just being obstructionist. Threat of a lawsuit usually solves such nonsense.

Good luck!