If all you're catching are small fish, then other locations should hold larger fish. Lure size usually doesn't determine fish size especially when it comes to panfish though very small panfish usually require a smaller hook and lure size. Yesterday I caught a fish size range of between 4" - 6" including a 1.5 lb bass using small lures rigged on 1/32 and 1/24 oz ball head jigs with #6 hook size.