Yeah probably not a great idea. What you CAN do is light the candle, then use wooden sticks like skewers to light the bowl, by lighting the end of the stick with the candle flame, taking a big hit, and then putting the flaming stick out in a container of sand or something. I use sand to extinguish incense sticks so I'm always set up for when a lighter runs out.

When I was camping recently we lost the lighter right in the middle of smoking; we ended up lighting bowls with pine-needles that we stuck in the fire lol