What is the law in your state regarding the patient-caregiver relationship?
If your caregiver is profiting off your card and not even returning your calls, you have every right to revoke his right to use your card. Skeezy bastard. I wouldn't just show up because someone slimy enough to pull that shit might also be slimy enough to fuck with you. Tell him you need to have better communication if he is going to continue to use your card, and then give him a couple days to shape up.

Like a sleeping mastiff, a grower at his grow is a dangerous creature to surprise.