You could let in a bird or a bat, which will eat the bees but then you would need to get rid of that.

Plug the hole they're using to get in.