Take it down based on who sketched out you feel.
How old is your brother? Is he old enough to really understand the potential consequences of his actions?
You need to sit him down without anger, as an older brother, and have a honest talk to him about family loyalty and the consequences that you would suffer if anything bad came of it. He probably never even considered the possibility that by yapping, he'd be putting you in danger.
I have to say this, though: We see yet another example of why you shouldn't try to grow in a house occupied by anyone who isn't involved wiht the grow personally....