Maybe as you're rolling its getting loose. Also, I find that if you grind the bud too fine it is harder to roll. I usually grind it so its still fine, but the grains are bigger, if you get what I'm saying. Other than that its just practice, practice, practice. Or you could just give up and by yourself a roller.