Oil temp is critical and also don't put too many in at once. Adding them too fast lowers the oil temp. Also, I use this trick on chicken and I bet it would work on fish too. After you fry it to almost the desired done-ness, take it out, let it cool and drain. Then pop it back in the hot oil for a few. Makes my chicken super crispy. haven't tried it with fish though.