Went back out today after work and talked to a guy at Fish USA for a bit. He also said more than likely it's sunfish and to slow down my hook set. I used the same set up I've been using on my 4.6 UglyStik and slowed down my hook set and that helped land a couple, I also went down to a 1/16th oz jig head with much smaller hook and that helped too. I also put that same set up on my 8ft BnM jigging pole and I could feel a lot more, didn't hook up much, I was trying not to get mowed over by jet skis and boats lol