From what I've experienced and one test done on the subject (I'll find the link to the study and repost it) driving high doesn't actually harm your ability to do normal driving, but if a situation came up (Such as a car about to hit you, or other unusual situations), you won't be able to react as quickly.

So you're probably safe if you're driving a few blocks from one house to another, but I'd recommend not driving high in any area that will take extra attention.