I'll fillet if the carcass is going into making a stock. Like slipknot mentioned, too much work on small fish and cooking off the bone takes some flavour away.