I'm in the same boat. I also judge from the lake. Some lakes just hold better quality fish than others. A giant in one lake might be average in another. Just depends on if I am out fishing for a meal or out fishing for bigger fish and being selective. If I'm for a meal, the lake doesn't matter. If I'm going for bigger fish, I might not even think about hitting one of those smaller fish lakes.
In the end, they all taste great. I don't have an issue cleaning any of them. Smaller fish I tend to scale and then fillet (not skinning) to save a little more meat. Bigger ones, I fillet and skin.

