you could have one bank on the charger that is bad, it's a trial and error/process of elimination in finding the problem. I would use a DVM to get readings on both batts, go fishing and do everything you normally would, when you get home, use the DVM again. This should tell you or point you in one direction to pursue, either the charger is bad, a battery is bad, etc. Go from there. If you know someone who has a portable charger that works that you could borrow to charge the batts up, take your DVM readings, then go fishing, take your DVM readings again, then use YOUR OB charger, then take readings again, that may help.
GO BIG ORANGE !
I meant to behave, but there were just way too many other options available at the time.