I think it's just your mind playing tricks on you. I used to feel that way a lot too when I first started. Another mind trick that is similar to that is when you walk into a room when you are really stoned and it seems like everyone is looking at you and/or everyone knows you are stoned out of your mind.

It's just your brain being paranoid, thats it.