Sharpstone grinder problem.

http://gotnorice.no-ip.org/grindermagnet.jpg

I have a sharpstone grinder (See pic/link above). The base and lid both have a small magnet that is normally covered by a thin layer of silver-ish metal. Well in my case, part of that thin layer of silver metal seems to have flaked off, and exposed the black/dark-grey magnet underneath. This happened on both the lid and the base. Since the surface of the magnet is kind of rough, they grind against each other and now every time I open my grinder there is a bit of black dust, looks like graphite dust, around where the two magnets contact each other.

I'm wondering if I should keep using the grinder or not? Presumably some of this magnet dust is mixing with my weed and I'm smoking it. I'm not really sure what possible health implications there might be.